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Abstract 

In  the  past  few  years,  Army  aviation  accidents  have  been  on  the  rise,  due  largely 
to  increases  in  mission  frequency  and  complexity,  and  diminishing  resources.  The 
magnitude  of  the  resulting  losses  (casualties,  dollars,  equipment)  has  prompted  the 
Commanding  General  of  the  Army  Safety  Center  to  demand  a  complete  examination  of 
the  way  safety  hazards  and  subsequent  safety  controls  are  evaluated  and  selected.  This 
project  integrates  value  focused  thinking,  Monte  Carlo  simulation,  and  integer 
programming  in  response  to  this  demand  by  developing  and  using  a  methodology  that 
effectively  identifies  and  evaluates  portfolios  of  controls.  An  integer  program  generates 
portfolios  of  controls  that  maximize  the  reduction  of  hazards  that  contribute  to  Army 
aviation  accidents.  Monte  Carlo  simulation  using  the  bootstrap  method  is  used  to 
simulate  the  number  and  types  of  losses  resulting  from  accidents  that  occur  in  100,000 
UH-60  flying  hours.  A  value  model  has  been  developed  to  quantify  the  severity  of  these 
losses.  The  expected  performance  of  the  portfolios  of  controls  is  calculated  as  die 
anticipated  decrease  in  severity  of  losses  resulting  from  implementation  of  those  controls. 


ANALYSIS  OF  UH-60  BLACKHAWK  SAFETY  CONTROLS 
USING  VALUE  FOCUSED  THINKING  AND  MONTE  CARLO  SIMULATION 

1  Introduction 

1.1  Background 

Management  of  safety  in  the  Army  starts  at  the  top,  with  the  Secretary  of  the 
Army  and  the  Chief  of  Staff,  Army.  Safety  policy  generated  from  this  level  is  actively 
supported  and  promoted  by  all  Army  staff  agencies.  The  Army  operates  “one  of  the 
largest,  most  comprehensive  safety  programs  in  the  world”  (US  Army  Safety  Center 
Mission  Statement).  The  USASC  provides  support  to  the  Army  staff  and  major 
subordinate  commanders,  assisting  them  in  conserving  manpower  and  materiel  resources 
and  in  conducting  effective  operations.  These  programs  are  designed  to  help  produce 
safe  air  and  ground  operations,  as  well  as  to  promote  safety  both  on  and  off  duty  (USASC 
Mission  Statement). 

In  the  past  few  years,  Army  aviation  safety  mishaps  have  been  on  the  rise,  due 
largely  to  the  increases  in  mission  frequency  and  complexity,  coupled  with  diminishing 
resources  (ASIST  Brief,  Aug  1999).  In  November  1998,  the  Under  Secretary  of  Defense 
for  Acquisition  and  Technology  challenged  the  services  to  “. .  .achieve  a  3  sigma 
reduction  in  Class  A  accident  rate  in  5  years”  (ASIST  Brief,  Aug  1999).  In  February 
1999,  the  Army  Safety  Action  Team  reviewed  Army  aviation  safety  experiences  in 
response  to  this  challenge,  and  chartered  the  Army  Safety  Investment  Strategy  Team 
(ASIST).  ASIST’s  strategy,  endorsed  by  the  Vice  Chief  of  Staff,  Army  (VCSA)  in  April 
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1999,  is  to  reduce  by  50%,  within  the  next  ten  years,  the  rate  of  fatal  and  disabling 
injuries,  the  Class  A-C  accident  rate,  and  the  total  annual  cost  of  aviation  accidents 
(ASIST  Brief,  Aug  1999). 

1.2  Risk  Management 

Approximately  eleven  years  ago,  risk  management  was  formally  introduced  in  the 
Army  safety  community,  and  has  since  evolved  into  the  principal  risk-reduction  process 
to  protect  the  force.  By  integrating  risk  management  into  all  Army  processes,  lives  can 
be  saved  and  equipment  spared.  Risk  management  “is  the  process  of  identifying, 
assessing,  and  controlling  risks  arising  from  operational  factors  and  making  decisions  that 
balance  risk  costs  with  mission  benefits”  (FM  100-14, 1998: 1-1).  Risk  has  two 
components:  probability  and  severity.  Probability  refers  to  the  chance  of  something  bad 
happening,  or  the  chance  of  a  hazard  causing  an  event,  while  severity  deals  with  the 
magnitude  of  consequences.  Accidents  are  unplanned  events  causing  personal  injury  or 
illness,  or  property  damage.  Historically,  accidents  have  caused  more  losses  than  has 
enemy  action  (FM  100-14,  1998:  1-2).  Proper  application  of  risk  management 
techniques  will  result  in  conservation  of  combat  power  and  resources  (FM  100-14, 1998: 
1-1). 

1.3  Problem  Statement 

The  purpose  of  this  project  is  to  develop  a  methodology  aimed  at  evaluating 
controls  that  reduce  UH-60  Blackhawk  helicopter  hazards.  A  hazard  is  defined  as  “any 
actual  or  potential  condition  that  can  cause  injury,  illness,  or  death  of  personnel,  damage 
to  or  loss  of  equipment,  property  or  mission  degradation”  (FM  101-14,  1998:  G-l). 
Controls  are  proposed  actions  intended  to  eliminate  or  reduce  the  severity  and/or 
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probability  of  a  hazard  contributing  to  an  accident  (FM  100-14, 1998:  2-13).  ASIST 
developed  the  UH-60  hazard  taxonomy  and  control  listing  in  their  proof  of  concept  study. 
This  project  builds  on  ASIST’s  initial  work  with  the  UH-60  program. 

1.4  Research  Objectives 

In  the  past,  the  United  States  Army  Safety  Center  (USASC)  used  statistical 
analysis  of  data  to  identify  safety-related  hazards  and  controls.  Some  consider  such  an 
analysis  approach  to  be  too  reactive.  Using  value-focused  thinking  and  multiattribute 
preference  theory  concepts,  Captain  Brian  J.  Sperling,  GOR/99M,  US  Army,  developed  a 
prescriptive  decision  analysis  model  to  more  proactively  evaluate  hazards  (Sperling, 
1999). 

The  focus  of  this  project  is  to  use  decision  and  risk  analysis  theory  and 
techniques,  coupled  with  simulation  and  mathematical  programming,  to  further  develop  a 
suite  of  tools  to  assist  Army  leadership  with  integrating  risk  management  and  Army 
processes. 

The  ASIST  effort  brought  safety  and  aviation  experts  together  to  focus  on 
eliminating  Army  aviation  hazards.  The  initial  project  was  limited  to  the  UH-60 
Blackhawk,  with  the  intent  of  expanding  out  to  all  Army  aviation  platforms.  This  project 
will  integrate  and  augment  ASIST’s  work  in  developing  a  methodology  for  designing  and 
proposing  portfolios  of  controls  intended  to  reduce  or  eliminate  UH-60  Blackhawk 
hazards. 

1.5  Research  Approach 

The  first  part  of  the  project  involves  updating,  verifying,  and  validating  the 
Accident  Severity  model.  Using  the  updated  Severity  of  Losses  Model,  a  severity  score 
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representing  the  losses  due  to  accidents  in  100,000  flight  hours,  is  determined.  A 
mathematical  program  uses  the  ASIST  control  list  and  hazard  taxonomy  to  generate 
portfolios  of  controls  for  evaluation.  Once  accomplished,  portfolios  of  controls  designed 
to  reduce  UH-60  hazards  can  be  assigned  an  expected  performance  score.. 

Updating  included  incorporating  changes  and  additions  to  the  hierarchy.  The 
Accident  severity  model  was  designed  for  all  Army  aircraft.  For  every  aspect  of  the 
model,  the  value  hierarchy  was  reassessed  to  make  sure  the  structure  was  still  valid  for 
the  UH-60.  The  ranges  for  each  level  of  evaluation  consideration  were  determined.  The 
single  dimensional  value  functions  were  updated  to  reflect  changes  in  perspective, 
preferences,  and  policy.  Weights  for  all  evaluation  criteria  and  measures  were  confirmed 
or  reassessed. 

Verification  assures  the  model  is  built  properly  by  comparing  the  conceptual 
model  to  the  computer  implementation.  It  confirms  the  model  properly  represents  all 
input  parameters  and  structures  by  assuring  the  logic  of  the  model  is  translated  correctly 
into  the  computer  program  (Banks,  et  al.:  399). 

Validation  is  the  determination  that  a  model  is  accurately  representing  the  real 
world  systems  (Banks,  et  al.:  400).  In  this  case,  validation  involves  assuring  the  model  is 
complete  and  nonredundant.  Kirkwood  (1997)  states  the  conditions  for  a  complete 
hierarchy  as,  ’’the  evaluation  considerations  at  each  layer  (tier)  in  the  hierarchy,  taken 
together  as  a  group,  must  adequately  cover  all  concerns  necessary  to  evaluate  the  overall 
objective  of  the  decision”  (Kirkwood,  1997:16).  In  addition  to  completeness,  the 
hierarchy  should  also  be  nonredundant. 
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A  mathematical  program  has  been  developed  for  the  purpose  of  generating 
portfolios,  with  the  intent  of  identifying  a  group  of  controls  that  meet  the  50%  reduction 
goals  at  minimal  investment  costs.  Once  the  portfolios  were  generated,  the  Monte  Carlo 
simulation  was  run  to  determine  the  performance  of  the  portfolios  of  controls  associated 
with  the  UH-60  program.  The  hazard  taxonomy  and  hazard  assessments  from  ASIST, 
along  with  the  control  effectiveness  assessments,  are  incorporated  into  the  model  at  this 
point.  Sensitivity  analysis  on  the  weight  of  the  major  uncertainty  in  the  model  was 
conducted  to  see  how  varying  the  weight  affects  the  portfolio  rankings. 

1.6  Scope 

The  UH-60  Blackhawk  was  chosen  as  the  focus  of  ASIST’ s  initial  study  because 
of  the  maturity  of  the  system  and  the  massive  amount  of  accident  data  available.  This 
project  was  intended  to  parallel  ASIST’ s  initial  study  and  is  therefore  limited  to  the  UH- 
60.  In  the  future,  the  methods  and  model  from  this  project  could  be  modified  for  other 
platforms.  This  project  deals  with  only  class  A,  B,  and  C  accident  categories.  Class  A 
accidents  involve  damages  of  more  than  $1M,  and/or  fatality  or  permanent  total 
disability.  Class  B  accidents  result  in  damages  between  $200K  and  $1M,  and/or 
permanent  partial  disability.  Class  C  accidents  have  damages  between  $10K  and  $200K 
or  loss  of  time  from  work  (AR  385-40, 1994:2.2-2.4).  Class  D  and  E  accident  categories 
have  insufficient  data  for  meaningful  analysis. 

1.7  Thesis  Overview 

Chapter  Two  contains  a  brief  description  of  Value  Focused  Thinking  (VFT), 
Monte  Carlo  simulation,  and  the  bootstrap  method,  along  with  some  references  to  recent 
applications  of  these  techniques.  Chapter  Three  describes  the  Accident  Severity  Model, 
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the  Monte  Carlo  simulation  methodology,  and  a  discussion  of  the  mathematical  program 
used  in  developing  the  control  portfolio.  Chapter  Four  contains  a  description  of  the  data, 
results,  and  sensitivity  analysis  of  selected  weights  from  the  model.  Chapter  5  discusses 
the  project  findings  and  recommendations  for  further  related  study.  Data  and  other 
relevant  information  are  included  in  the  Appendices. 
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2  Literature  Review 


2.1  Introduction 

This  chapter  provides  a  review  of  the  literature  pertinent  to  this  research  project  as 
well  as  an  introduction  to  basic  decision  analysis  concepts.  The  discussion  will  also 
include  the  basic  concepts  of  value  focused  thinking,  which  are  used  extensively  in  this 
research  project.  Monte  Carlo  simulation  and  the  bootstrap  method  are  briefly  discussed. 
Finally,  current  applications  using  these  techniques  are  reviewed. 

2.2  Decision  Analysis 

2.2.1  Decision  Analysis  Concepts 

Decision  analysis  is  a  methodology  used  to  assist  a  decision  maker  in  modeling  a 
given  decision  situation,  to  include  the  structure  of  the  problem,  and  his  or  her  beliefs  and 
preferences.  Decision  analysis  is  a  prescriptive  approach,  designed  to  help  intelligent 
people  make  difficult  decisions  (Clemen,  1996:  3).  By  using  decision  analysis 
techniques,  real-world  problems  can  be  analyzed  in  order  to  gain  insight  and 
understanding  (Clemen,  1996:  xix). 

Decisions  can  be  difficult  due  to  a  number  of  factors.  A  decision  may  be 
considered  hard  because  of  its  complexity  or  its  uncertainty.  By  using  decision  analysis, 
the  problem  can  be  broken  into  manageable  pieces  and  organized  into  an  understandable 
structure.  Decision  analysis  techniques  can  also  help  identify  and  represent  key 
uncertainties  in  a  useful  manner  (Clemen,  1996:  2-3).  Decisions  can  also  be  difficult 
when  there  are  multiple,  competing  objectives,  or  when  different  perspectives  lead  to 
different  decisions.  Decision  analysis  “provides  both  a  framework  and  specific  tools  for 
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dealing  with  multiple  objectives”  (Clemen,  1996:  4).  When  different  perspectives  lead  to 
different  decisions,  decision  analysis  can  help  resolve  these  differences  in  a  systematic 
way  (Clemen,  1996:  4) 

By  using  decision  analysis  techniques,  a  thorough  understanding  of  the  problem  is 
formed  and  a  better  decision  can  be  made  (Clemen,  1996:  3).  It  is  vital,  however,  to 
understand  that  a  better  decision  does  not  necessarily  result  in  a  better  outcome.  Not 
every  detail  can  be  captured  and  incorporated  into  a  decision  situation,  and  occasionally 
unpredictable  events  happen.  A  good  decision  might  be  considered  choosing  an 
alternative  that  ‘provides  the  ideal  outcome  given  eventual  circumstances’,  or  one  that 
comes  ‘from  a  thorough  understanding  of  the  problem’  (Bunn,  1984:  9). 

Decision  analysis  does  not  provide  solutions.  Instead,  it  is  “best  thought  of  as 
simply  an  information  source,  providing  insight  about  the  situation,  uncertainty, 
objectives,  and  trade-offs,  and  possibly  yielding  a  recommended  course  of  action” 
(Clemen,  1996:  4).  Decision  analysis  does  not  replace  the  decision  maker.  Rather,  it 
works  with  the  decision  maker  to  help  him  or  her  thoroughly  understand  the  problem. 

The  responsibility  for  the  decision,  however,  ultimately  remains  with  the  decision  maker 
(Clemen,  1996:  4). 

2.2.2  Decision  Analysis  Proces  s 

The  decision  analysis  process  applies  the  fundamental  concepts  in  a  systematic 
manner  (see  Figure  2.1).  In  each  step,  certain  questions  can  be  asked  to  help  guide  the 
process.  The  deliverables  represent  the  type  of  results  usually  derived  from  a  particular 
step. 
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Questions 


What  do  we  want? 
What  do  we  know? 
What  can  we  do? 


What  are  the 
relationships? 
What  is  important? 


What  are  the  possible 
outcomes? 

Probability  Outcomes? 

How  much  could 
we  gain/lose? 


Can  we  decide 
OR  how  much 
more  information 
would  we  be 
willing  to  pay  for? 


Iteration 


Deliverables 


Values 

Information 
Alternatives 
Influence  Diagram 


Business  Model  Probability  Distributions  Value  of  Information 
Sensitivity  Analysis  Dominated  Alternatives  Value  of  Control 
Critical  Uncertainties  Risk  Profiles 


Figure  2-1  Decision  Analysis  Cycle  (Clemen,  1996:6) 

In  structuring  the  problem,  the  goal  is  an  understanding  of  the  nature  of  the 
decision  as  well  as  the  objectives  in  the  current  situation.  In  the  next  step,  deterministic 
analysis,  the  problem  is  decomposed  into  manageable  subproblems.  Initial  sensitivity 
analysis  can  help  identify  key  uncertainties  and  influential  parameters.  In  the 
probabilistic  step,  these  uncertainties  are  incorporated  into  the  model.  The  final  step 
helps  determine  if  a  decision  is  ready  to  be  made,  or  if  further  analysis  is  required 
(Clemen,  1996:  6-7). 

Solving  problems  using  a  decision  analysis  approach  is  an  iterative  process.  As 
the  model  is  developed,  new  insight  could  require  a  restructuring  of  the  problem.  In 
addition,  at  each  step  along  the  way,  the  decision  maker’s  preferences  could  change,  or 
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more  information  could  become  available,  which  would  require  a  restructuring  of  the 
problem  (Clemen,  1996:  7). 

When  dealing  with  complex  decisions  involving  multiple  competing  objectives, 
reaching  a  conclusion  can  be  difficult,  especially  when  trying  to  account  for  the 
complexities  involved  with  uncertainty,  dependencies,  and  tradeoffs.  When  problems 
become  complex,  augmenting  intuition  with  decision  analysis  methodologies  can  assist  in 
making  logical,  supportable,  and  transparent  decisions  (Kirkwood,  1997:  1). 

2.3  Value  Focused  Thinking 

Hazards  contribute  to  accidents  that  can  cause  loss  of  life  or  irreparable  damage  to 
equipment.  Many  interrelated  factors  might  be  considered  when  evaluating  controls 
designed  to  eliminate  hazards.  Value  focused  thinking  is  an  approach  designed  to  deal 
with  complex  decisions  having  multiple,  competing  objectives,  that  involve  tradeoffs;  it 
is  quite  different  from  the  alternative  based  way  of  addressing  a  problem.  The  most 
common  way  of  attacking  a  complex  problem  is  to  focus  on  the  available  alternatives. 
This  alternative  based  approach,  which  is  reactive  rather  than  proactive,  places 
unnecessary  limitations  on  the  decision  situation.  Value  focused  thinking  removes  the 
artificial  boundaries  and  allows  creative  thinking  to  augment  the  structuring  of  the 
decision  problem  (Keeney,  1992:  viii). 

“Values  are  what  we  care  about”  (Keeney,  1992:  3).  Values  are  also  things  that 
are  considered  important  in  a  decision  situation.  Given  this,  values  should  ultimately 
drive  the  decision.  Objectives  represent  the  “preferred  direction  of  movement”  with 
respect  to  the  important  aspects  of  the  situation  (Kirkwood,  1997:  12).  Once  the  set  of 
values  is  obtained,  it  can  be  organized  into  a  hierarchy  with  the  overarching  objective  at 
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the  top,  and  supporting  objectives  and  measures  of  merit  at  subsequent  levels  (see  Figure 
2-2).  The  measures  of  merit,  or  evaluation  considerations,  measure  the  attainment  of  the 
higher  objectives. 

One  of  three  approaches  is  typically  used  to  identify  and  incorporate  values  into  a 
value  hierarchy.  These  are  the  gold,  silver,  and  platinum  standards  (Parnell,  et  al,  1 998). 
The  gold  standard  development  is  a  top-down  method  that  clarifies  and  structures 
doctrine  or  strategic  vision.  Using  strategic  visions,  doctrine,  or  other  organizational 
documents  can  facilitate  ‘buy-in’  to  the  process.  Each  objective  is  divided  into  sub¬ 


objectives,  and  each  sub-objective  is  further  divided  in  successively  greater  detail,  until  a 
measurable  item  is  reached  (Kirkwood,  1992:  21-22). 


Figure  2-2  Sample  Value  Hierarchy 

It  is  not  always  possible  to  use  the  gold  standard  development  method.  In 
situations  where  strategic  objectives  or  doctrine  are  not  known,  or  when  the  alternatives 
are  already  known,  the  silver  standard  development  may  be  appropriate.  The  silver 
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standard  development  method  is  a  bottom-up  approach  that  assists  in  creating  new 
values.  By  using  verbs  that  indicate  action  to  describe  the  situation,  a  group  of 
representatives  can  inductively  develop  a  value  hierarchy.  This  method  typically  requires 
a  large  group  of  people  and  a  substantial  amount  of  time  to  develop  the  hierarchy 
(Parnell,  et  al,  1998). 

The  third  method  is  the  platinum  standard.  In  cases  where  the  problem  is  ill 
defined,  the  platinum  standard  can  help  build  the  value  hierarchy,  inductively,  through  a 
series  of  interviews  with  the  decision  makers,  representatives,  and  stakeholders. 
Combining  the  information  gleaned  from  the  interviews  and  existing  doctrine  can  help 
establish  the  true  nature  of  the  problem  (Parnell,  et  al,  1998). 

Value  hierarchies,  regardless  of  the  method  used  to  develop  them,  should  possess 
certain  qualities.  According  to  Kirkwood,  these  qualities  should  be  completeness, 
nonredundancy,  decomposability,  operability,  and  small  size. 

Completeness  of  a  hierarchy  is  a  quality  signifying  that  all  evaluation  concerns 
required  to  evaluate  the  higher  objective  are  taken  into  account.  It  also  means  all 
evaluation  concerns  together  “adequately  cover  all  concerns  necessary  to  evaluate  the 
overall  objective  of  the  decision”  (Kirkwood,  1992:  16).  A  complete,  or  collectively 
exhaustive,  value  hierarchy  is  needed  to  properly,  and  consistently,  distinguish  between 
different  alternatives  (Kirkwood,  1997:  17) 

A  value  hierarchy  should  also  be  nonredundant,  or  mutually  exclusive.  No 
evaluation  consideration  should  overlap  another  in  the  same  level  of  the  hierarchy.  This 
means  no  two  measures  should  be  used  in  evaluating  the  degree  of  attainment  for  the 
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same  objective.  Nonredundancy  is  needed  to  prevent  double  counting  of  an  evaluation 
consideration  (Kirkwood,  1997:  16-17) 

Decomposability,  or  independence,  requires  that  the  preference  for  one  level  of  an 
evaluation  consideration  should  not  depend  on  the  level  of  another  evaluation 
consideration.  The  lack  of  decomposability  makes  the  model  much  more  complicated 
and  difficult  to  use  (Kirkwood,  1997: 17-18) 

“An  operable  value  hierarchy  is  one  that  understandable  for  the  persons  who  must 
use  it”  (Kirkwood,  1997:  18).  Compromises,  with  respect  to  the  other  desirable  qualities 
of  a  hierarchy,  must  sometimes  be  made  in  order  to  make  the  hierarchy  operable.  These 
compromises  are  certainly  acceptable  as  long  as  the  decision  maker  believes  the 
hierarchy  to  be  complete  and  understandable. 

All  things  being  equal,  the  smaller  the  hierarchy,  the  better.  A  smaller  hierarchy 
is  easier  to  communicate,  and  also  requires  fewer  resources  to  evaluate  the  alternatives 
(Kirkwood,  1997:  18). 

Having  built  the  hierarchy,  evaluation  measures  must  also  be  developed.  The 
scales  for  evaluation  measures  are  first  classified  as  natural  or  constructed: 

•  Natural  scale  -  is  in  general  use  with  a  common  interpretation 

•  Constructed  scale  -  is  developed  for  a  specific  decision  situation,  and 
generally  used  when  a  natural  scale  is  not  appropriate  or  does  not  exist 

Evaluation  measure  scales  are  further  classified  as: 

•  Direct  scale  -  directly  measures  degree  of  attainment  of  an  objective 

•  Proxy  scale  -  uses  the  attainment  of  an  associated  objective  to  measure  the 
degree  of  attainment  for  the  original  objective  (Kirkwood,  1997:  24). 
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The  following  example  contains  the  types  of  evaluation  measures. 


Natural 

Constructed 

Direct 

Net  Present  Value 
Time  to  Remediate 
Cost  to  Remediate 
Bandwidth  per  second 

Olympic  Diving  Score 
Weather  Prediction  Categories 
Project  Funding  Categories 

R&D  Project  Categories 

Proxy 

Gross  National  Product 
(Economic  Growth) 
Number  of  Subsystems 
(System  Reliability 

Performance  Evaluation  Categories 
(Promotion  Potential) 

Student  Grades 
(Student  Learning) 

Figure  2-3  Evaluation  Measure  Types  (Parnell,  et  al,  1998) 


For  a  deterministic  analysis,  single  dimensional  value  functions  are  developed  for 
each  evaluation  measure.  The  value  functions  are  created  in  a  systematic  manner  so  that 
they  reflect  the  decision  maker’s  and/or  organizations  values  or  beliefs.  From 
interviewing  the  decision  maker,  or  a  representative,  the  value  for  a  certain  score  for  an 
evaluation  measure  is  elicited  by  asking  value  increment  questions  (Kirkwood,  1997:  70). 
After  covering  the  entire  range  of  the  evaluation  measure,  the  value  function  represents 
how  the  decision  maker  feels  about  a  given  score. 

For  decisions  involving  tradeoffs  among  the  objectives,  the  decision  maker’s 
preferences  are  included  in  the  model  by  assigning  weights  to  each  evaluation  measure 
(Kirkwood,  1997:  55).  Once  the  weights  are  determined,  the  overall  additive  value 
function,  which  represents  the  weighted  sum  of  single  dimensional  values,  can  be 
created. 

2.4  The  Bootstrap  Method 

The  bootstrap  method  is  a  technique  that  uses  computer  intensive  methods  in 
making  reliable  statistical  inferences,  such  as  standard  errors,  confidence  intervals,  or 
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other  measures  of  uncertainties  (Davison  and  Hinkley,  1997:  2).  Using  the  bootstrap 
method,  random  samples  are  drawn,  with  repetition,  from  one  or  more  empirical 
distributions.  In  complex  situations  where  bootstrap  statistics  are  awkward  to  compute, 
Monte  Carlo  samples  can  be  used  to  approximate  the  parameters  of  interest  (Efron  and 
Tibshirani,  1993:  1). 

The  bootstrap  method  is  useful  in  situations  when  there  is  no  known  well-defined 
probability  distribution.  Key  advantages  of  using  the  bootstrap  method  are  that  it  is  easy 
to  describe  and  apply  to  complicated  situations,  and  that  distribution  assumptions,  such  as 
normality,  are  not  needed  (Efron  and  Tibshirani,  1993:  160).  Precautions  must  be  taken, 
however,  to  ensure  appropriate  design  of  the  experiment,  data  analysis,  and  presentation 
of  conclusions.  Failing  to  do  so  could  lead  to  either  solving  the  wrong  problem,  or 
solving  the  right  problem  poorly  (Davison  and  Hinkley,  1997:  4). 

2.5  Monte  Carlo  Simulation 

Monte  Carlo  simulation  consists  of  some  physical  or  mathematical  system  that 
can  be  described  in  terms  of  probability  distribution  functions.  As  a  rule,  the  algorithm  is 
to  simply  complete  a  single  random  trial.  Then,  the  single  random  trial  is  repeated  n 
times,  each  independent  of  the  other  (Sobol,  1984:  10).  The  final  goal  of  the  Monte 
Carlo  method  is  to  gather  pertinent  statistics  of  desired  metrics  based  on  the  distribution 
of  the  n  independent  outcomes  (Hammersby  and  Handscomb,  1964:  10). 

The  Monte  Carlo  method  heavily  employs  the  use  of  randomly  generated 
numbers.  Each  random  number  used  in  the  system  is  a  potential  source  of  uncertainty  in 
the  final  result.  Attempts  should  be  made  to  replace  uncertainties  with  exact, 
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theoretically  sound  representations  whenever  possible  (Hammersby  and  Handscomb, 
1964:  5). 

Monte  Carlo  simulation  can  be  conducted  in  a  variety  of  manners.  One  way  is 
using  the  Crystal  Ball  add-in  package  for  Microsoft  Excel.  Leaving  the  modeling  of  the 
system  to  the  user,  Crystal  Ball  provides  common  probability  distributions  than  can  be 
used  in  the  simulation.  Crystal  Ball  also  gathers  the  results  from  each  trial  of  the 
simulation  and  reports  statistics  upon  termination.  Microsoft  Excel  and  Crystal  Ball  are 
used  extensively  in  this  project. 

2.6  Portfolio  Analysis 

At  various  levels,  government  agencies  and  businesses  are  forced  to  make 
decisions  regarding  resource  allocation.  Two  common  procedures  used  in  making  these 
decisions  are  ranking  schemes  and  mathematical  programming.  Ranking  processes  tend 
to  be  loosely  structured,  while  mathematical  programming  is  designed  to  find  an  optimal 
solution  given  an  objective  and  a  set  of  constraints  (Byrd  and  Moore,  1982: 183).  The 
knapsack,  a  specific  type  of  mathematical  program,  is  used  in  this  study. 

The  knapsack  problem  is  based  on  the  premise  that  a  hiker  would  like  to  take  as 
many  items  as  possible  on  a  trip,  but  can  only  carry  a  specific  weight  in  his  or  her 
knapsack.  The  idea  is  to  choose  items  that  maximize  the  total  benefit  while  remaining 
under  the  weight  restriction  (Winston,  1994:  468).  This  approach  can  be  used  to  identify 
the  best  mix  of  programs  to  fund  that  would  give  the  maximum  benefit  for  a  specified 
investment  budget  (Byrd  and  Moore,  1982:  184). 
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2.7  Applications  of  Techniques  in  Current  Literature 

This  section  discusses  a  few  applications  of  value  focused  thinking,  as  well  as  a 
recent  application  of  Monte  Carlo  simulation  using  the  bootstrap  method. 

Capt  Brian  K.  Sperling,  GOA-99M,  developed  a  decision  analysis  model  for  the 
US  Army  Safety  Center.  Using  value  focused  thinking,  Capt  Sperling  developed  an 
accident  severity  model,  incorporating  Army  doctrine,  commander  preferences,  and 
expert  opinion.  The  model  determined  severity  for  accident  and  hazard  categories. 

Using  probability  relationships,  accident  and  hazard  risks  were  also  quantified.  This 
research  effort  builds  on  Capt  Sperling’s  work. 

The  CEO  of  British  Columbia  Hydro  and  Power  Authority  (BC  Hydro),  a 
publicly  owned  hydroelectric  utility  company,  had  a  vision  of  making  BC  Hydro  the  best 
planned  electric  utility  in  North  America.  Ken  Peterson,  appointed  as  the  director  of 
planning,  found  little  guidance  in  the  published  mission  statement  on  how  to  attack  this 
problem.  He  realized  he  needed  to  identify  the  organization’s  strategic  objectives  in 
order  to  achieve  the  long-term  goal  (Keeney,  McDaniels,  1992). 

Using  a  value  focused  thinking  approach,  Keeney  and  McDaniels  helped 
Peterson  develop  a  hierarchy  of  the  company’s  strategic  objectives.  Information,  from 
various  points  of  view,  was  gathered  on  BC  Hydro  strategic  objectives  through  a  series  of 
interviews  with  three  key  individuals.  The  nature  of  the  interviews  was  unstructured  and 
open,  so  as  not  to  place  any  arbitrary  limitations.  Questions  by  the  analysts,  like  “what 
does  that  objective  really  mean?”,  helped  clarify  relationships  among  the  objectives. 

After  a  few  follow-on  meetings,  a  strategic  objectives  hierarchy  was  developed  and 
utility  functions  were  developed.  BC  Hydro  continued  to  use  this  multiobjective 
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structure  in  other  contexts  such  as  capital  budget  reductions  and  developing  an  its 
integrated  electricity  plan  (Keeney,  McDaniels,  1992) 

In  a  thesis  by  Capt  Donald  F.  Hurry,  GOR-96M,  various  statistical  techniques 
were  used  to  determine  the  impact  of  Programmed  Depot  Maintenance  on  weapon  system 
availability.  Specifically,  the  technique  of  Monte  Carlo  simulation  with  the  bootstrap 
method  was  used  to  estimate  the  downtime  per  truck  per  year.  By  independently 
sampling,  with  replacement,  from  two  empirical  distributions,  time  to  failure  and 
downtime  after  failure  was  estimated.  These  estimates  validated  conclusions  drawn  from 
other  statistical  analyses. 
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3  Methodology 


3.1  Introduction 

“The  mission  of  the  Army  Safety  Center  is  to  enhance  combat  readiness  through 
proactive  risk  management  to  prevent  accidents”  (USASC  Mission  Statement).  The 
USASC  accomplishes  this  by  assisting  the  Army’s  major  commands  and  the  Army  staff 
with  the  development  and  day-to-day  management  of  safety  policies.  Commanders  then 
execute  those  policies  and  procedures  at  the  unit  level  (USASC  Mission  Statement). 

A  current  tasking  is  to  “...achieve  a  3  sigma  reduction  in  Class  A  accident  rate  in 
5  years”  (ASIST  Brief,  Aug  1999).  The  Army  Safety  Investment  Strategy  Team  (ASIST) 
was  chartered  in  response  to  this  challenge.  ASIST’s  strategy  is  to  reduce  by  50%, 
within  the  next  ten  years,  the  rate  of  fatal  and  disabling  injuries,  the  annual  cost  of 
aviation  accidents,  and  the  Class  A-C  Accident  Rate.  Expanding  on  ASIST’s  work,  this 
research  incorporates  risk  management  into  the  process  by  using  a  value  focused  thinking 
approach  to  tie  together  doctrine,  commander  preferences,  expert  opinion,  and 
information  from  the  Risk  Management  Information  System. 

3.2  Project  Overview 

The  overarching  objective  of  this  project  is  to  reduce  losses  due  to  aviation 
accidents.  For  this  study,  which  is  limited  to  the  UH-60  program,  the  interpreted 
objective  is  to  identify  and  implement  a  methodology  that  identifies  and  evaluates 
portfolios  of  controls  intended  to  reduce  the  severity  of  losses  by  50  percent  over  the  next 
ten  years. 
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The  performance  of  a  control  is  estimated  by  simulating  the  effect  on  losses  due 
to  implementing  that  control  (see  Figure  3-1).  The  Severity  of  Losses  Model,  based  on 
value  focused  thinking,  quantifies  the  severity  of  losses  due  to  UH-60  accidents,  while 
the  Monte  Carlo  process  accounts  for  probability  in  the  model.  Using  the  Severity  of 
Losses  Model  and  Monte  Carlo  simulation,  accidents  that  occur  in  100,000  flying  hours, 
with  and  without  controls,  are  simulated.  Control  performance  is  then  defined  as  the 
expected  decrease  in  losses  after  implementation  of  controls.  Using  the  control 
performance  and  cost  estimates,  portfolios  of  controls  are  identified  that  meet  Army 
Safety  goals. 


This  chapter  contains  a  description  of  the  Severity  of  Losses  Model,  to  include 
what  it  represents,  its  construction,  data  aggregation,  and  how  ASIST’s  hazard  taxonomy. 
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hazards  assessments,  and  controls  assessments  were  implemented.  The  Monte  Carlo 
simulation,  using  the  bootstrap  method,  is  described  in  detail.  Finally,  the  mathematical 
program  used  to  generate  portfolios  is  discussed. 

3.3  Severity  of  Losses  Model 

This  model  quantifies  the  severity  of  losses  resulting  from  accidents  in  100,000 
flying  hours.  Accidents  are  unplanned  events  causing  personal  injury  or  illness,  or 
property  damage.  Hazards  are  any  actual  or  potential  condition  that  can  lead  to  an 
accident.  For  the  purpose  of  analysis,  it  is  assumed  that  reducing  the  frequency  or 
severity  of  contributing  hazards  should  reduce  the  overall  losses  associated  with  the 
accidents. 

The  Severity  of  Losses  Model,  incorporating  expert  opinion  and  commander 
preferences,  quantifies  the  losses  from  UH-60  accidents.  The  components  of  the  model 
are  supported  in  Army  doctrine  and  are  representative  of  the  important  aspects  in 
evaluating  hazards  and  controls  (Warren,  2000).  FM  100-14,  Army  Risk  Management, 
states  four  criteria  to  consider  when  assessing  severity  (FM  100-14, 1998:  2-9).  The  top 
tier  of  the  Severity  of  Losses  hierarchy  represents  degree  of  injury  or  illness,  other 
mission  impairing  factors,  repair  or  replacement  costs,  other  mission  impairing  factors, 
and  environmental  damage.  Casualties  represents  the  degree  of  injury  or  illness,  unit 
readiness  represents  other  mission-impairing  factors,  and  total  costs  represents  loss  of  or 
damage  to  equipment  or  property.  The  top  tier  of  the  hierarchy  is  shown  in  Figure  3-2. 
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Figure  3-2  Top  Tier  Severity  of  Losses  Hierarchy 


The  current  Severity  of  Losses  Model,  based  on  the  Accident  Severity  Model 
developed  by  Capt  Brian  Sperling  (Sperling,  1999),  quantifies  the  severity  of  losses 
resulting  from  accidents  in  100,000  flying  hours,  rather  than  from  a  single  accident  or 
hazard.  This  change  of  viewpoint  necessitated  modification  of  the  Accident  Severity 
model.  While  the  top  tier  of  the  hierarchy  is  outlined  in  FM  100-14  (see  Figure  3-2),  the 
questions  of  why  and  to  whom  are  they  important  required  investigation.  Discussions 
with  members  of  ASIST  and  USASC  brought  out  the  answers  to  these  questions. 

Accidents  affect  the  Army  as  a  whole,  but  an  accident  also  impacts  the  owning 
unit.  The  effects  of  the  severity  of  losses  from  casualties  are  felt  at  both  the  Army  level 
and  at  the  unit  level.  Similarly,  the  loss  of  an  aircraft  costs  the  Army  millions  of  dollars, 
but  perhaps  even  more  severe  is  the  decrease  in  unit  readiness  resulting  from  having 
fewer  aircraft  for  training  and  mission  execution  (Semmens,  2000). 

To  represent  these  two  levels,  the  evaluation  measures  in  each  tier  focus  on  the 
appropriate  level  in  Army  command.  Casualties ,  total  costs,  and  environmental  damage 
represent  the  severity  of  losses  at  an  Army  level  perspective.  Unit  readiness  looks  at  the 
impact  of  losses  on  the  battalion.  This  distinct  difference  in  the  level  of  focus  accounts 
for  concerns  at  both  the  battalion  unit  and  Army  levels  of  command. 


22 


3.3.1  Evaluation  Measures 


This  section  describes  the  sub-criteria  of  the  Severity  of  Losses  Model.  Each  top 
level  consideration  is  further  broken  down  into  evaluation  considerations.  As  noted  in 
Figure  2-3,  the  evaluation  measures  are  classified  as  either  natural-direct,  natural-proxy, 
constructed-direct,  or  constructed-proxy.  The  metrics  used  to  measure  the  evaluation 
considerations  are  also  discussed. 

3.3.1. 1  Casualties 

The  casualties  branch  of  the  hierarchy  measures  the  severity  contribution  of 
fatalities  and  injuries  sustained  in  accidents.  These  considerations  take  an  Army  level 
perspective  of  determining  severity.  According  to  Army  Risk  Management,  FM  100-14, 
and  USASC  leadership,  the  following  evaluation  considerations  effectively  measure 
casualties’’  contribution  to  severity  (Warren,  2000). 


Figure  3-3  Severity  of  Losses:  Casualties 

3.3.1. 1.1  Loss  of  Life 

Loss  of  life  is  a  natural-direct  measure  that  assesses  the  severity  of  the  number  of 
fatalities  in  100,000  flying  hours.  In  this  measure,  all  civilian  and  military  deaths  caused 
by  UH-60  Blackhawk  accidents  are  included. 
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3.3.1. 1.2  Permanent  Disabilities 

There  are  two  types  of  permanent  disabilities  tracked  in  the  Risk  Management 
Information  Database  (RMIS):  total  and  partial.  Total  disabilities  compromise  any 
nonfatal  injury  that,  in  the  opinion  of  competent  medical  authority,  permanently  and 
totally  incapacitates  a  person  to  the  extent  that  he  or  she  cannot  find  any  gainful 
employment  (DA  PAM  385-40,  1994:  Glossary).  The  occurrence  of  this  type  of 
disability  resulting  from  aviation  accidents  is  extremely  rare.  No  permanent  total 
disabilities  were  found  in  the  UH-60  five-year  database,  and,  therefore,  are  not  used  in 
this  study. 

Permanent  partial  disability  is  a  natural-direct  measure  used  to  assess  the 
contribution  to  severity  of  losses  from  permanent  disabilities.  This  type  of  injury  occurs 
frequently  enough  to  warrant  inclusion  in  the  model.  A  permanent  partial  disability  is 
any  injury  (not  resulting  in  death  or  permanent  total  disability)  that,  in  the  opinion  of 
competent  medical  authority,  results  in  the  loss  or  permanent  impairment  of  any  part  of 
the  body,  with  the  following  exceptions: 

a.  Loss  of  teeth. 

b.  Loss  of  fingernails  or  toenails. 

c.  Loss  of  tip  of  fingers  or  tip  of  toe  without  bone  involvement. 

d.  Inguinal  hernia,  if  it  is  repaired. 

e.  Disfigurement 

f.  Sprains  that  do  not  cause  permanent  limitation  of  motion  (DA  PAM  385-40, 
Glossary). 

3.3.1. 1.3  Time  Incapacitated 

Time  incapacitated  is  a  natural-direct  measure  used  to  assess  the  total  number  of 
days  of  hospitalization  resulting  from  aviation  accidents.  Hospitalization  is  defined  as 
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“admission  to  a  hospital  as  an  inpatient  for  medical  treatment”  (DA  PAM  385-40, 1994: 
Glossary). 


3.3.1.2  Unit  Readiness 

The  effects  of  losses  due  to  accidents  are  felt  at  both  the  Army  and  unit  level. 
Unit  Readiness  measures  the  severity  of  losses  in  terms  of  impact  on  the  battalion.  The 
Unit  Readiness  branch  of  the  hierarchy  is  shown  in  Figure  3-4.  Training  execution,  unit 
morale,  and  equipment  availability  were  identified  as  the  major  contributing  factors  to 
unit  readiness  (Semmens,  2000). 


Figure  3-4  Severity  of  Losses:  Unit  Readiness 


3.3.1.2.1  Training  Execution 

Training  execution  is  a  natural-proxy  scale  that  assesses  the  battalion 
commander’s  change  in  mission  planning  and  execution  resulting  from  the  number  and 
class  of  recent  accidents.  The  rationale  is  that,  as  more  accidents  occur  in  a  unit,  the 
commander  is  believed  to  become  more  risk  averse.  As  a  result,  the  commander  may 
‘pull  back’  on  the  amount,  complexity,  realism,  or  sophistication  of  training,  which 
reduces  unit  readiness.  LTC  Semmens  expressed  the  opinion  that  the  reduction  in 
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training  execution  due  to  recent  accidents  is  most  often  the  major  cause  of  a  decrease  in 
unit  readiness  (Semmens,  2000). 

3.3.1.2.2  Unit  Morale 

A  decline  in  unit  morale  also  has  a  detrimental  effect  on  unit  readiness.  After 
discussing  the  many  issues  involved  in  assessing  the  impact  on  unit  morale  from  aircraft 
accidents,  two  major  influences  were  identified:  loss  of  life  and  magnitude  of  injury.  Loss 
of  life  is  a  natural-proxy  measure  that  assesses  the  decrease  in  morale  in  terms  of  the 
number  of  fatalities  in  the  unit.  Magnitude  of  injury  uses  a  constructed  scale  as  a  proxy 
measure  to  assess  the  effect  on  unit  morale  (Semmens,  2000). 

3.3.1.2.3  Equipment  Availability 

Equipment  availability  is  a  natural-proxy  measure  to  assess  the  impact  on  unit 
readiness  resulting  from  fewer  aircraft.  An  unavailable  aircraft  is  defined  as  any  aircraft 
requiring  greater  than  40  man-hours  of  repair  time,  or  a  total  loss  of  an  aircraft.  The 
assumption  is  that  repairs  greater  than  40  hours  would  not  be  accomplished  at  the  unit 
level,  requiring  the  aircraft  to  be  fixed  at  the  post  or  depot  level  (Semmens,  2000).  If  an 
aircraft  is  not  economically  repairable,  it  is  considered  a  total  loss  (AR  385-40, 1994:  2- 
11). 

Man-hours  to  repair  are  at  first  estimated  in  the  accident  report,  then  updated  upon 
completion  of  the  work.  The  final  man-hour  total  includes  time  to  estimate  damage, 
repair  and  replace  damaged  components,  and  remove  and  replace  parts  not  economically 
repairable  (AR  385-40,  1994:  2-11). 
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3.3.1.3  Total  Costs 


Total  costs  is  a  natural-direct  measure  that  assesses  the  severity  of  accidents  in 
terms  of  the  dollar  cost  to  the  Army.  Army  accident  costs  are  computed,  as  describe  in 
Army  Regulation  735-11,  using  injury  costs,  repair/replacement  costs,  and  other  military 
and  non-military  damage  costs  resulting  from  accidents. 

All  aircraft  damage  costs  are  accounted  for  in  repair/replacement  cost.  When  the 
aircraft  is  a  total  loss,  the  aircraft  acquisition  cost  is  used.  When  the  aircraft  is  repairable, 
the  replacement  cost  is  computed  using  the  actual  cost  of  replacement  parts  and  man-hour 
costs  (AR  385-40, 1994:  2-1 1). 

Total  Costs 

—  Repair/Replacement  + 

Injury  Costs  + 

Damage  Costs 

Figure  3-5  Severity  of  Losses:  Total  Costs 
Injury  costs  are  used  in  calculating  total  accident  cost,  although  they  are  not  used 

in  determining  accident  classifications.  Injury  costs  include  the  cost  of  pay  while  away 

from  work,  medical  treatment,  hospitalization,  dependent  survival,  unused  training  costs, 

gratuities,  compensation,  disability  retirement,  and  burial.  Injury  costs  do  not  include 

indirect  costs  associated  with  the  accident  such  as  wages  lost  to  employees  not  injured 

(production  loss)  or  cost  of  hiring  and  training  new  employees.  The  actual  amount  of 

time  away  from  work  may  not  be  known  at  the  time  the  accident  report  is  submitted.  If  it 

is  not  known,  an  official  estimate  of  lost  workdays,  made  by  a  competent  medical 

authority,  is  used  in  computing  the  cost  (AR  385-40, 1994:  2-1 1). 
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3.3.1.4  Environmental  Damage 

The  Army  Risk  Management  manual  identifies  environmental  damages  as  a 
necessary  criterion  in  assessing  severity.  FM  20-400,  the  Military  Environmental 
Protection  Manual,  states  the  applicable  sub-criteria  given  in  Figure  5-2.  However,  little 
data  currently  exists  to  allow  a  detailed  assessment  of  environmental  damage. 


Figure  3-6  Severity  of  Losses:  Environmental  Damage 
Constructed-direct  measurements  are  used  in  assessing  soil  and  water  damage. 

The  Army  categorizes  hazardous  fluid  spills  based  on  the  number  of  gallons  spilled  (see 

Table  3-7).  In  assessing  the  severity  of  environmental  damages,  effect  on  the 

environment  and  the  amount  of  cleanup  required  are  taken  into  account. 

3.3.2  Severity  Functions 

This  section  provides  a  description  of  the  process  used  to  develop  single 
dimensional  severity  functions  for  each  evaluation  measure.  Single  dimensional  value 
functions  are  usually  constructed  so  that  the  least  preferred  level  receives  a  zero,  and  the 
most  preferred  level  receives  a  one  (Kirkwood,  1997:  61).  In  assessing  the  severity  of  the 
evaluation  measures,  the  most  preferred  level  is  the  least  severe  level.  Therefore  the  least 
preferred  level  of  the  evaluation  receives  a  severity  of  one  (Sperling,  99:  3-26). 
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The  expected  range  of  each  metric  is  determined  through  simulation  based  on 
accident  data  from  FY94  through  FY98.  The  range  represents  the  lower  and  upper 
bounds  that  are  needed  in  developing  evaluation  measures.  Two  different  procedures 
were  used  in  building  the  severity  functions.  In  creating  a  piecewise  linear  severity 
function,  relative  severity  increments  between  the  possible  evaluation  measure  scores 
were  specified  (Kirkwood,  1997:  62).  Exponential  severity  functions  are  used  where  the 
severity  increments  are  not  linear.  In  these  functions,  a  mid-severity  score  and  its 
associated  measure  score  were  specified.  The  mid-severity  score  is  the  score  in  which 
the  difference  in  severity  between  the  lowest  score  and  the  mid-severity  score  is  the  same 
as  the  difference  in  severity  between  the  mid-severity  score  and  the  highest  score 
(Kirkwood,  1997:  66). 

The  severity  functions  were  developed  through  interviews  with  appropriate 
representatives  from  the  USASC.  Casualties  and  total  costs  severity  functions  were 
developed  with  COL  Warren,  USASC  Deputy  Commander.  Unit  readiness  severity 
functions  were  developed  with  a  recent  battalion  commander,  LTC  Semmens,  USASC 
Executive  Officer.  Finally,  environmental  damage  severity  functions  were  developed 
with  an  expert  in  environmental  law,  LTC  Gleisberg,  USASC  Judge  Advocate  General. 

3.3.2.1  Casualties 

The  following  severity  functions  assess  the  severity  of  casualties  occurring  in 
100,000  UH-60  Blackhawk  flying  hours,  from  an  Army  level  perspective. 
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3.3.2. 1.1  Loss  of  Life 


The  loss  of  any  individual  is  tragic  and  could  never  be  adequately  quantified. 
However,  from  an  Army  perspective,  the  loss  of  each  additional  life  due  to  aviation 
accidents  is  equally  severe,  resulting  in  a  linear  function  (Warren,  2000).  From  fiscal 
year  1994  through  1998,  the  average  number  of  fatalities  per  100,000  flying  hours  was 
about  2.53.  The  maximum  number  occurring  in  a  single  accident  in  that  timeframe  was 
eight.  Results  from  1 0,000  simulation  runs  suggest  an  expected  range  of  0  to  27  fatalities 
in  100,000  UH-60  flying  hours. 


Figure  3-7  Severity  Function:  Loss  of  Life 

3.3.2.1.2  Permanent  Partial  Disabilities 

Similar  to  loss  of  life ,  the  effect  of  permanent  partial  disabilities  is  also  tragic.  In 
terms  of  severity  to  the  Army,  each  occurrence  is  equally  severe  (Warren,  2000).  Again, 
a  linear  severity  function  is  used.  The  only  occurrence  in  the  accident  database  of 
permanent  partial  disabilities  was  a  single  accident  with  two  permanent  partial 
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disabilities.  Simulation  results  suggest  an  expected  range  of  0  to  6  permanent  partial 


disabilities. 


Number  of  Partial  Disabilities 

Figure  3-8  Severity  Function:  Partial  Disabilities 

3.3.2.1.3  Time  Incapacitated 

Time  incapacitated  is  scored  in  terms  of  total  days  in  the  hospital  as  a  result  of 
UH-60  accidents  in  100,000  flying  hours.  A  linear  severity  function  is  used  because, 
from  an  Army  perspective,  each  day  of  hospitalization  is  equally  severe  (Warren,  2000). 
Using  past  accident  data  and  the  Monte  Carlo  simulation  an  expected  range  of  time 
incapacitated  is  0  to  52  days  of  hospitalization  (see  Figure  3-9). 

3.3.2.2  Unit  Readiness 

Unit  readiness  focuses  on  the  battalion  level.  The  severity  of  losses  in  each 
measure  is  for  each  individual  unit,  but  is  representative  of  100,000  total  UH-60  flying 
hours,  or  roughly  six  months  for  the  current  UH-60  program.  The  information  used  in 


31 


developing  the  severity  functions  for  training  execution,  unit  morale,  and  equipment 
availability  was  elicited  from  LTC  Semmens,  a  recent  aviation  battalion  commander. 


Figure  3-9  Severity  Function:  Time  Incapacitated 

Throughout  the  interview,  he  emphasized  that  more  aviation  commanders  need  to  be 
interviewed  to  better  capture  the  nature  of  these  severity  functions.  Because  of  this,  the 
severity  functions  used  in  this  study  are  based  upon  guidance  from  LTC  Semmens  rather 
than  explicit  data  points. 

3.3.2.2.1  Training  Execution 

Training  execution  assesses  the  impact  on  unit  readiness  resulting  from  a 
reduction  in  training  frequency,  complexity,  and  realism.  This  measure  combines  both 
the  number  and  class  of  accidents  occurring  in  a  single  unit  within  a  range  of  0  to  3 
accidents.  This  range  was  determined  through  simulation  and  is  an  realistic  representation 
(Semmens,  2000).  From  the  battalion  commander’s  viewpoint,  the  effect  on  risk  aversion 


32 


is  about  the  same  for  three  Class  C  accidents  as  it  is  for  one  Class  A  accident.  The 

following  scale,  designed  to  account  for  the  class  and  the  number  of  accidents,  is  used  in 

measuring  training  execution: 

Class  A  accident  =  3 
Class  B  accident  =  2 
Class  C  accident  =  1 

For  example,  if  a  unit  had  two  Class  A  accidents  in  the  time  span  of  100,000  UH-60 
flying  hours  (approximately  six  months),  the  training  execution  category  would  be  six. 
Two  Class  Cs  and  one  Class  B  accident  would  be  a  Category  4  (Semmens,  2000).  The 
largest  increase  in  severity  occurs  between  two  and  four  on  the  accident-class  category 
scale.  After  a  single  Class-A,  or  two  Class-B  or  -C  accidents,  commanders  may  become 
more  risk-averse  and  ‘pull  back’  on  the  amount  of  flying.  Severity  increases  slowly  after 
category  six,  representing  a  threshold  above  which  the  unit  would  most  likely  have 
already  reduced  flying  dramatically  (Semmens,  2000). 
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Figure  3-10  Severity  Function:  Training  Execution 
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3.3.2.2.2  Unit  Morale 


A  decrease  in  unit  morale  can  affect  unit  readiness.  For  the  purpose  of  this 
analysis,  unit  morale  is  subdivided  into  lives  lost  in  unit  and  magnitude  of  injury. 

3.3.2.2.2.1  Lives  Lost  in  Unit 

Constructing  a  severity  function  for  lives  lost  in  unit  is  complicated  because  each 
commander  will  have  a  different  viewpoint  on  the  matter.  LTC  Semmens  had  experience 
in  developing  severity  functions  and  was  familiar  with  the  concept  of  evaluating 
increments  in  severity.  In  the  interview,  LTC  Semmens  was  shown  a  curve  with  a  range 
of  0  to  9  and  a  midvalue  of  3.  He  felt  that  a  unit  could  cope  with  the  first  few  fatalities, 
after  which  each  additional  lost  life  increasingly  affects  the  unit.  The  lives  lost  in  unit 
severity  function  described  by  LTC  Semmens  is  shown  in  Figure  3-11.  Nine  fatalities  in 
a  single  unit  during  a  six-month  timespan  would  be  a  rare  occurrence,  but  possible.  This 
range  was  determined  through  Monte  Carlo  simulation  using  UH-60  accident  data  for 
fiscal  years  1994  through  1998.  The  severity  represents  the  impact  on  unit  morale 
because  of  the  fatalities  rather  than  the  value  or  importance  of  lives  lost. 
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Figure  3-1 1  Severity  Function:  Lives  Lost  in  Unit 

3.3.2.2.2.2  Magnitude  of  Injury 

Magnitude  of  injury,  shown  in  Figure  3-12,  uses  the  following  scale  in  assessing 
severity: 


Category  0: 
Category  1 : 
Category  2: 

Category  3: 


No  injuries  requiring  hospitalization 

Injuries  requiring  less  than  7  days  hospitalization 

Injuries  requiring  more  than  7  days  hospitalization,  but  no 

permanent  disabilities 

Injuries  resulting  in  permanent  disabilities 


Category  1  represents  minor  injuries,  which  rarely  affect  the  morale  of  the  unit. 
Unit  morale  is  affected  slightly  more  when  personnel  sustain  more  serious  injuries, 
represented  by  Category  2.  Permanent  disabilities,  Category  3,  are  considered  much  more 
severe  in  comparison  and  tend  to  have  a  lingering  effect  in  the  minds  of  unit  personnel 
(Semmens,  2000). 


35 


1 


0.875  • 
0.75  - 
0.625  - 

«  0.5  • 

4> 

(0  0.375  - 
0.25 
0.125  - 

o4 

0 


♦ 


♦ 

♦ 

1  2  3 

Category 


Figure  3-12  Severity  Function:  Magnitude  of  Injury 

33.2.2.3  Equipment  Availability 

The  number  of  unavailable  aircraft  in  a  unit  is  defined  as  any  aircraft  requiring 
more  than  40  hours  of  maintenance,  or  not  economically  repairable,  as  a  result  of  an 
accident.  Using  past  accident  data  and  Monte  Carlo  simulation,  the  expected  range  for 
the  number  of  unavailable  aircraft  in  a  unit  was  determined  to  be  0  to  5  unavailable 
aircraft. 

The  severity  function  in  Figure  3-13  shows  a  major  increase  in  severity  between 
two  and  three  unavailable  aircraft.  This  threshold  represents  the  level  where  a  typical 
unit  would  begin  to  feel  the  impact  of  fewer  aircraft  (Semmens,  2000). 
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Number  of  Aircraft 


Figure  3-13  Severity  Function:  Aircraft  Availability 

3.3.2.3  Total  Costs 

Total  Costs  assesses  the  severity  of  UH-60  accidents  in  terms  of  dollar  cost  to  the 
Army.  Repair/replacement,  injury,  and  damage  costs  in  100,000  flying  hours  are  totaled. 
Looking  at  the  range  of  costs  generated  by  simulation  using  past  accident  data,  0.1  to  52 
million,  there  is  no  threshold  that  warrants  a  significant  jump  in  severity  (Warren,  2000). 
When  considering  the  fact  that  the  UH-60  is  only  one  fleet  of  aircraft  in  the  Army 
inventory,  this  makes  even  more  sense.  Therefore,  the  total  costs  severity  function  is 
linear. 
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Total  Accident  Costs  ($M) 


Figure  3-14  Severity  Function:  Total  Costs 

3.3.2.4  Environmental  Damage 

Specifically,  this  measure  uses  the  maximum  occurrence  of  spill  category  in 

100,000  flying  hours.  Based  on  Army  Regulation  385-40,  fuel,  hydraulic  fluid,  and  oil, 

fluid  spills  are  estimated  as  follows: 

Category  0:  No  hazardous  fluid  spilled 
Category  1 :  Less  than  1  gallon  of  hazardous  fluid  spilled 
Category  2:  More  than  1,  but  less  than  2  gallons 

Category  3 :  More  than  2,  but  less  than  1 0  gallons 

Category  4:  More  than  10,  but  less  than  20  gallons 
Category  5:  More  than  20  gallons  spilled 

For  example,  if  there  were  three  accidents  with  recorded  spillage  in  100,000  UH- 
60  Blackhawk  flying  hours,  the  maximum  spill  category  for  all  types  of  hazardous  fluids 
(fuel,  hydraulic  fluid,  and  oil)  from  the  three  accidents  would  be  used  in  the  severity 
function. 
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3.3.2.4.1  Soil  Damage 

Severity  of  damage  to  the  soil  is  assessed  in  amount  of  hazardous  fluids  spilled  in 
an  accident.  From  an  Army  perspective,  UH-60  hazardous  fluid  spills  are  generally  of 
little  concern.  In  considering  soil  damage,  there  is  little  difference  in  the  type  of 
hazardous  fluid  spilled.  Therefore,  the  same  severity  function  is  used  for  fuel,  hydraulic 
fluid,  and  oil.  The  severity  of  a  Category  5  spill  was  estimated  to  be  3  times  as  severe  as 
a  Category  4  (Gleisberg,  2000). 
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Figure  3-15  Severity  Function:  Soil  Damage 

3.3.2.4.2  Water  Damage 

Severity  of  damage  to  water  sources  is  dependent  on  the  type  of  fluid  spilled. 
Fuel  spills  tend  to  dissipate  easily,  except  in  large  quantities.  In  these  cases,  the  severity 
of  a  Category  5  spill  was  estimated  to  be  10  times  as  severe  as  a  Category  4.  Once  in  the 
water,  hydraulic  fluid  and  oil  tend  to  clump  near  the  surface.  For  these  types  of  water 
spills,  a  Category  5  is  judged  to  be  3  times  as  severe  as  a  Category  4  (Gleisberg,  2000). 
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Figure  3-16  Severity  Function:  Water  Damage  -  Fuel 


0.75 

£ 

®  0.5 

O 

CO 

0.25 

0 

0  1  2  3  4  5 

Fluid  Spill  Category 

Figure  3-17  Severity  Function:  Water  Damage  -  Hydraulic  and  Oil 
3.3.2.5  Severity  of  Losses  Hierarchy 

The  criteria  and  evaluation  considerations  in  this  were  validated  with  ASIST  and 
the  decision  maker’s  representative,  COL  Warren  (AFIT  Brief,  2000).  Further  research, 
if  warranted,  may  improve  upon  the  current  hierarchy  by  incorporating  changes  or 
additions  as  required.  Figure  3-18  displays  the  entire  hierarchy. 
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3.3.3  Weight  Assessment 

Before  describing  the  process  used  to  determine  the  weights  for  the  Severity  of 
Losses  Model,  it  is  necessary  to  review  the  subtle  difference  between  value  functions  and 
severity  functions.  Single  dimensional  value  functions  are  usually  constructed  so  that  the 
least  preferred  level  receives  a  zero,  and  the  most  preferred  level  receives  a  one.  In 
assessing  the  severity  of  the  evaluation  measures,  the  most  preferred  level  is  the  least 
severe  level.  For  this  reason  the  least  preferred  level  of  the  evaluation  receives  a  severity 
of  one. 


Aircraft 


Figure  3-18  Severity  of  Losses  Hierarchy 
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The  severity  scores  from  each  evaluation  measure  are  combined  to  generate  an 
overall  severity  of  losses  score.  One  way  of  combining  the  severity  scores,  perhaps  the 
simplest,  might  be  to  average  them.  In  most  situations  involving  tradeoffs,  however, 
average  scores  fail  to  take  into  account  any  tradeoffs  that  may  exist.  Attaching  weights  to 
the  evaluation  measures  alleviates  this  problem  (Kirkwood,  1997:59). 

Weights,  normally  elicited  from  the  decision  maker  or  a  designated 
representative,  represent  the  importance  placed  over  the  range  of  each  evaluation 
measure.  Combining  the  weights  and  the  severity  functions  yields  the  overall  additive 

n 

severity  function  (Kirkwood,  1 997 :  230):  severity  of  losses(X)  =  ^  wtst(xi) ,  where  X 

<=i 

represents  the  set  of  attributes  under  consideration,  w  is  the  weight,  and  s( )  is  severity. 

3.3.3.1  Local  Weights 

Local  weights,  derived  from  pairwise  comparisons  of  evaluation  measures, 
represent  the  relative  importance  of  each  measure  in  a  single  branch  of  a  hierarchy 
(Kirkwood,  1997:  71).  Using  the  ranges  for  each  evaluation  measure,  the  local  weights 
for  each  branch  of  the  hierarchy  were  elicited  from  the  appropriate  experts  and 
representatives. 

3.3.3.1.1  Local  Weights  -  Casualties 

Weights  in  the  casualties  branch  of  the  hierarchy  were  assessed  using  information 
elicited  from  COL  Warren.  Using  hypothetical  situations  representing  extreme 
situations,  he  was  able  to  compare  the  severity  of  measurable  ranges  from  evaluation 
measures  and  establish  their  relative  importance  (Kirkwood,  1997:  71). 
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The  first  hypothetical  situation  involved  having  a  control  that  eliminated  all  losses 
due  to  accidents  except  for  loss  of  life,  which  remained  at  its  most  severe  level  of  52 
fatalities.  Next,  he  was  asked  to  imagine  a  different  control  that  eliminated  all  losses 
except  for  permanent  disabilities,  which  remained  at  its  most  severe  level  of  6  permanent 
partial  disabilities.  When  asked  which  control  he  would  rather  implement,  COL  Warren 
chose  the  second.  This  response  indicated  that,  within  the  given  ranges,  loss  of  life 
should  be  weighted  more  than  permanent  disabilities.  A  similar  situation  comparing 
permanent  disabilities  with  time  incapacitated  resulted  in  a  greater  emphasis  placed  on 
permanent  disabilities. 

Using  the  same  hypothetical  situations,  COL  Warren  was  next  asked  to  determine 
a  level  on  the  loss  of  life  scale  with  which,  when  compared  with  the  highest  level  of 
permanent  partial  disabilities,  he  would  be  indifferent  between  the  two  controls.  The 
level  of  loss  of  life  stated  was  two.  One  way  to  view  this  statement  is  that  the  severity 
increment  of  going  from  no  fatalities  to  two  is  equal  to  the  severity  increment  of  going 
from  no  partial  disabilities  to  six.  In  comparing  the  severity  of  permanent  disabilities 
with  time  incapacitated,  the  level  of  indifference  for  the  permanent  partial  disability 
measure  was  three  disabilities.  These  relationships  result  in  the  following  equations: 

H'C ightlcns  of  life  *  Severltyioss  of  life(2)  =  WCZg'/z^permanent disabilities 

Weight  loss  of  life  +  Weight  permanent  disabilities  ~f"  Weight  time  incapacitated  —  1 

Weigh  ^permanent  disabilities  ^  .S'CVCV'/f^'permanent  disabilities  (3  )  —  WClgHtt  ime  incapacitsted 

Solving  this  group  of  equations  results  in  the  following  local  weights: 
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Figure  3-19  Local  Weights  -  Casualties 
3.3.3.1.2  Local  Weights  -  Unit  Readiness 

In  determining  the  weights  for  unit  readiness,  LTC  Semmens,  the  most  recent 
aviation  battalion  commander  at  the  Army  Safety  Center,  again  recommended 
interviewing  more  battalion  commanders  to  attain  a  more  diverse  representation.  Rather 
than  give  explicit  severity  comparisons,  he  provided  general  guidance  on  the  nature  of  the 
weights.  On  the  upper  level,  he  said  if  equipment  availability  was  given  a  one,  then  unit 
morale  would  be  a  two,  and  training  execution  would  be  a  three.  These  answers 
represented  ratios  of  severity  increments,  but  he  stressed  that  these  were  guides  for 
weights.  Within  unit  morale,  he  stated  that,  at  the  unit  level,  magnitude  of  injuries  have  a 
slightly  higher  effect  than  fatalities.  His  estimation  was  about  a  1 .2  to  1  ratio  of 
magnitude  of  injury  to  lives  lost  (Semmens,  2000).  A  note  was  made  to  conduct 
sensitivity  analysis  of  these  weights  as  part  of  the  overall  analysis.  These  estimations 
result  in  the  following  local  weights  for  unit  readiness: 
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Figure  3-20  Local  Weights  -  Unit  Readiness 

3.3.3.1.3  Local  Weights  Environmental  Damage 

In  interviewing  LTC  Gleisberg,  hypothetical  situations  similar  to  those  presented 
to  COL  Warren  were  used  to  determine  the  local  weights  of  the  evaluation  measures.  For 
soil  damage,  the  increment  in  severity  going  from  a  Category  0  to  a  Category  5  hydraulic 
fluid  spill  is  equivalent  to  going  from  a  Category  0  to  a  Category  4  fuel  spill.  Similarly, 
the  increment  in  severity  from  a  Category  0  to  a  Category  5  oil  spill  is  equivalent  to 
going  from  a  Category  0  to  a  Category  4  fuel  spill. 

For  water  damage,  LTC  Gleisberg  commented  that  an  oil  spill  is  much  more 
damaging  than  a  fuel  spill.  Fuel  dissipates  rapidly  in  water,  while  oil  tends  to  clump  and 
stick  to  rocks  and  wildlife.  In  comparing  oil  with  fuel,  going  from  a  Category  0  to  a 
Category  5  fuel  spill  is  equivalent  in  severity  to  going  from  a  Category  0  to  a  Category  3 
oil  spill.  Hydraulic  fluid  spills  in  water  are  worse  than  fuel,  but  not  as  bad  as  oil 
(Gleisberg).  The  increment  in  severity  from  a  Category  0  to  a  Category  5  hydraulic  spill 
is  equivalent  in  severity  of  going  from  a  Category  0  to  a  Category  4  oil  spill  in  terms  of 
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water  damage.  In  comparing  the  severity  of  soil  damage  versus  water  damage,  LTC 
Gleisberg  stated  that  going  from  a  Category  0  to  a  Category  5  fuel  soil  spill  is  equivalent 
in  severity  to  going  from  a  Category  0  to  a  Category  5  oil  spill  in  water.  Using  these 
relationships  the  following  local  weights  were  derived: 


Figure  3-21  Local  Weights  -  Environmental  Damage 

33.3.2  Global  Weights 

Having  determined  the  local  weights  in  each  branch  of  the  hierarchy,  the  higher 
level  weights  in  the  hierarchy  were  assessed  in  the  same  manner.  COL  Warren  made 
severity  comparisons,  at  Army  level,  between  evaluation  measures  in  different  branches 
of  the  hierarchy.  Similar  to  the  hypothetical  situations  used  in  assessing  local  weights,  he 
was  asked  to  imagine  controls  that  eliminate  all  losses  except  for  one,  which  remained  at 
its  most  severe  level. 
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The  first  hypothetical  situation  involved  selecting  between  a  control  that 
eliminated  all  losses  except  for  a  Category  5  hazardous  fluid  spill,  and  a  control  that 
removed  all  losses  except  for  $52  million  of  total  accident  costs.  COL  Warren  stated 
that,  in  this  hypothetical  situation,  he  would  choose  the  first  control.  For  him  to  be 
indifferent  between  the  two  controls,  the  level  would  have  to  be  $1  million  of  total  costs 
(Warren,  2000). 

The  next  hypothetical  situation  involved  a  control  that  removed  all  losses  except 
for  $52  million  of  total  accident  costs,  and  a  control  that  removed  all  losses  except  that  it 
left  training  execution  at  its  most  severe  level.  Training  execution  is  focused  at  the 
battalion  level,  while  total  costs  is  at  Army  level.  The  way  training  execution  is  defined, 
the  most  severe  level  would  be  three  Class-A  accidents  in  each  unit.  This  would  be 
equivalent  to  78  total  Class-A  accidents  in  100,000  flying  hours,  since  the  model  assumes 
there  are  26  UH-60  battalions.  COL  Warren  chose  the  first  control,  stating  that,  while 
$52  million  is  a  lot  of  money  to  the  Army,  it  does  not  compare  to  the  effect  on  training 
execution  that  many  accidents  would  have.  To  feel  indifferent  about  the  two  controls, 
training  execution  would  have  to  be  at  a  level  of  two.  This  level  would  be  equivalent  to 
each  unit  sustaining  a  single  Class-B  accident  in  100,000  flying  hours.  In  comparison,  an 
average  training  execution  level  of  2  across  all  26  battalions  is,  in  terms  of  severity, 
equivalent  to  $52  million  in  total  accident  costs  (Warren,  2000). 

The  next  hypothetical  situation  involved  a  control  that  removed  all  losses  except 
for  $52  million  of  total  accident  costs,  and  a  control  that  removed  all  losses  except  that  it 
left  loss  of  life  at  its  most  possible  severe  level  of  52  fatalities.  While  it  is  hard  to  place  a 
dollar  value  on  a  life,  this  type  of  tradeoff  must  be  made.  Looking  at  both  loss  of  life  and 
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total  accident  costs  from  an  Army  perspective,  COL  Warren  stated  he  would  rather 
implement  the  first  control.  For  him  to  feel  indifferent  between  the  severity  of  the  two 
controls,  the  level  of  loss  of  life  would  have  to  be  four.  Thus  at  an  Army  level  this  would 
mean  the  severity  of  four  fatalities  is  equivalent  to  the  severity  of  $52  million  in  total 
accident  costs.  It  is  recognized  that  other  key  leaders  may  express  different  tradeoff 
preferences,  which  would  then  require  the  top  portfolio  to  be  checked  for  sensitivity  to 
this  change. 

By  using  the  following  relationships,  as  well  as  the  previously  determined  local 
relationships,  the  global  weights  for  the  Severity  of  Losses  Model  were  calculated  (see 
Figure  3-22). 

Weight  Training  Execution  *  Severity  Training  Execution  (2)  =  Weight  Total  Accident  Costs 
Weight  Total  Accident  Costs  *  Severity  Total  Accident  Costs  (1)  =  Weight  fuel  soil  damage 
weight  loss  of  life  *  severity  loss  of  life(4)  =  weight  Total  Accident  Costs 
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Figure  3-22  Severity  of  Losses  Global  Weights 
The  weights  elicited  and  used  in  this  model  represent  the  values  and  preferences 

of  the  current  leadership  at  the  USASC  and  may  not  represent  the  official  US  Army 

preferences.  The  entire  Severity  of  Losses  Model  should  be  reassessed  periodically  to 

ensure  it  remains  representative. 

3.4  Hazards 

The  current  model,  as  described  above,  quantifies  the  severity  of  losses  resulting 
from  UH-60  accidents  occurring  in  100,000  flying  hours.  Controls  have  been  developed 
to  eliminate  or  reduce  the  effects  of  the  hazards  that  contribute  to  the  cause  or  severity  of 
the  accidents.  This  section  describes  the  hazard  taxonomy  and  the  hazard  contribution 
assessments  from  ASIST. 
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3.4.1  ASIST  Hazard  Taxonomy 

In  developing  the  new  hazard  taxonomy,  members  of  the  ASIST  team  conducted 
a  thorough  review  of  all  UH-60  accident  narratives.  Using  the  definition  that  a  hazard  is 
made  up  human,  materiel,  and  environmental  factors,  they  developed  a  detailed  list  of  67 
hazards,  which  is  included  in  Appendix  A.  This  taxonomy  represents  a  breakthrough  as  a 
focus  on  the  different  parts  of  the  man-machine-environment  system  that  worked  together 
to  produce  a  hazard.  Emphasis  was  taken  off  of  who  made  a  mistake  or  which  engine 
part  failed  in  favor  of  a  more  holistic  and  preventative  approach  (ASIST  Brief,  1999). 

3.4.1. 1  ASIST  Hazard  Assessments 

ASIST  estimated  hazard  contribution  assessments  for  three  areas:  casualty,  cost, 
and  frequency.  Casualty  refers  to  injuries  and  fatalities,  cost  refers  to  total  accident  costs, 
and  frequency  refers  to  the  cause  of  the  accident.  Each  assessment  estimates  the  percent 
contribution  of  the  hazard  in  an  accident  in  the  areas  of  casualty,  cost,  and  frequency. 

For  an  accident,  any  number  of  hazards  may  contribute  to  these  areas,  but  the  total 
contribution  of  all  hazards  in  each  area  must  be  100  percent.  Table  3-1  contains  an 
example  of  hazard  assessments  for  two  accidents. 
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Table  3-1  Hazard  Assessment 


Accident 

Contributing 

Hazards 

Casualty 

Contribution 

Cost 

Contribution 

Frequency 

Contribution 

AVN-05 

0.333 

0.333 

0.333 

19940304 

H60-05 

0.667 

0.667 

0.667 

Total 

1.000 

1.000 

1.000 

AVN-01 

0.364 

0.364 

0.400 

19951110 

AVN-03 

0.181 

0.181 

0.200 

H60-18 

0.364 

0.364 

0.400 

H60-20 

0.091 

0.091 

0.000 

Total 

1.000 

1.000 

1.000 

Accident  19940304  had  two  contributing  hazards.  Across  all  three  categories, 
AVN-05  was  responsible  for  1/3  of  the  accident  losses  and  H60-05  was  responsible  for 
2/3.  In  the  second  example,  even  though  H60-20  contributed  to  the  severity  of  the 
accident,  its  frequency  contribution  was  zero,  indicating  it  had  nothing  to  do  with  the 
actual  cause  of  the  accident. 

3.5  Controls 

Value  models  are  typically  used  to  evaluate  and  develop  alternatives.  In  this 
study,  controls  or  portfolios  of  controls,  are  the  alternatives.  ASIST  developed  a  list  of 
49  controls  designed  to  attack  various  hazards  in  the  UH-60  Blackhawk  program. 
Similar  to  the  hazard  contribution  assessments,  each  control  has  an  assessed  percent 
effectiveness  against  each  hazard.  Even  though  the  controls  were  developed  with  a 
particular  hazard  in  mind,  many  controls  affect  more  than  a  single  hazard.  Experienced 
Army  aviators  and  members  of  ASIST  spent  hundreds  of  hours  to  develop  the  control 
effectiveness  assessments  which  represent  the  estimated  hazard  reduction  at  the  end  of 
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ten-years,  and  apply  to  the  casualty,  cost,  and  frequency  areas  of  the  hazard  (ASIST 
Control  Calculations  Brief,  1999).  Investment  costs  for  the  ten-year  period  were  also 
estimated  (see  Table  3-2).  While  a  massive  and  time-consuming  effort,  ASIST’ s  efforts 
were  invaluable  to  this  thesis.  A  complete  control  listing,  including  description,  affected 
hazards,  and  estimated  investment  costs,  is  included  in  Appendix  B. 

Table  3-2  H60-C09  Control  Effectiveness 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 
H60-08  0.6 

H60-09  0.6 

H60-18  0.6 

H60-58  0.5 

3.6  Control  Evaluation  Using  Monte  Carlo  Simulation 

The  Severity  of  Losses  Model  quantifies  the  losses  from  accidents  occurring  in 
100,000  UH-60  flying  hours.  Since  the  Army  reports  the  rate  of  accidents  per  100,000 
flying  hours,  this  same  block  of  time  is  used  to  simulate  accident  losses. 

Control  performance  is  defined  as  the  decrease  in  expected  severity  resulting  from 
implementation  of  that  control.  This  is  evaluated  by  combining  the  control  effectiveness 
assessments  with  the  hazard  contribution  assessments  to  influence  the  frequency  and 
outcomes  of  the  accidents  in  a  100,000  flying  hour  operational  period. 

The  Monte  Carlo  simulation  approach  used  produces  a  distribution  of  expected 
severity  scores.  The  expected  performance  of  a  control,  or  group  of  controls,  is  defined 
as  the  expected  decrease  in  the  severity  of  losses  resulting  from  the  implementation  of  the 
controls. 
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Using  Monte  Carlo  and  the  bootstrap  method  helps  in  dealing  with  difficulties 
involved  in  randomly  generating  accident  data.  Since  there  is  no  accident  distribution  to 
draw  from,  past  accidents  are  used  as  the  type  of  UH-60  accidents  likely  to  occur  in  the 
future.  In  this  case,  bootstrap  sampling  accounts  for  correlation  of  accident  data.  As 
mentioned  in  chapter  two,  the  bootstrap  method  is  useful  in  situations  in  which  there  is  no 
well-defined  underlying  probability  distribution.  Using  the  UH-60  accident  database  is 
just  such  a  situation. 

3.6.1  Monte  Carlo  Simulation 

Monte  Carlo  simulation  and  the  bootstrap  method  are  used  to  generate  data  that 
represents  the  losses  that  might  occur  from  accidents  in  100,000  UH-60  flying  hours. 

The  methodology  consists  of  conducting  B  independent  samples,  with  replacement,  each 
consisting  of  n  data  points  (Efron  and  Tibshirani,  1993:  47).  Each  of  the  n  data  points 
represents  the  severity  of  losses  from  the  accidents  that  might  occur  in  100,000  flying 
hours. 

To  generate  a  single  data  point,  or  one  severity  score,  accidents  are  randomly 
drawn,  with  replacement,  from  an  empirical  distribution  of  accidents.  The  distribution  of 
accidents  used  in  this  study  are  all  Class  A-C  UH-60  accidents  that  occurred  in  fiscal 
years  1994  through  1998.  The  number  of  draws  in  each  replication  is  binomial  random 
variable  with  30  trials  and  p  =  0.29.  The  distribution  is  based  on  UH-60  Class  A-C 
accident  rates  from  the  same  five  years. 

The  data  set  used  in  the  simulation  represents  5  years  of  accident  data,  or  about  10 
blocks  of  100,000  flying  hours.  For  this  reason  n= 10  data  points  are  generated  in  each  of 
the  B  replications.  It  is  important  to  note  that  a  data  point  is  not  the  severity  of  an 


53 


accident  but  the  severity  of  a  collection  of  accidents  that  may  have  occurred  over  100,000 
flying  hours  of  the  UH-60.  This  simulation  was  implemented  using  Microsoft  Excel  and 
the  commercial  add-in  software  package  Crystal  Ball.  Crystal  Ball  uses  Monte  Carlo 
simulation  to  provide  forecasting  and  risk  analysis  capabilities.  A  description  of  the 
Monte  Carlo  simulation  is  included  in  Appendix  C. 

3.6.2  Accident  Rate  Adjustment 

The  effect  of  implementing  a  control  is  estimated  by  modifying  the  accident  data. 
The  expected  frequency  reduction  is  used  to  reduce  the  number  of  accidents  occurring  in 
the  simulation.  Data  from  the  randomly  selected  accidents  is  then  transformed  and 
aggregated  to  the  appropriate  level  for  input  to  the  Severity  of  Losses  Model.  This 
produces  a  single  score  representing  the  severity  of  losses  for  100,000  flying  hours.  The 
number  of  accidents  in  each  run  of  the  Monte  Carlo  simulations  is  a  random  number 
based  on  historical  accident  rates.  Controls  can  be  designed  to  reduce  the  probability  of  a 
hazard  contributing  to  an  accident  or  the  severity  of  hazards,  or  both.  The  simulation 
model  accounts  for  the  probabilistic  aspects  of  the  control  by  reducing  the  number  of 
accidents  in  the  run.  The  extent  of  the  reduction  of  the  raw  accident  data  is  determined 
by  calculating  the  casualty,  cost,  and  frequency  reduction  totals.  Each  control  has  a  list 
of  hazards  it  affects  along  with  the  estimated  percent  effectiveness.  Each  accident  has  a 
list  of  percent  contribution  of  each  hazard.  When  an  accident  is  selected  in  the  simulation, 
each  hazard  contribution  assessment  is  combined  with  the  control  effectiveness  estimate 
to  determine  the  amount  of  frequency  reduction. 
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3.6.3  Accident  Data  Transformation 

In  the  base  case,  fiscal  year  1994  through  1998  accident  data  is  simply  aggregated 
and  input  into  the  Severity  of  Losses  Model.  This  action  returns  the  base  case  expected 
severity,  which  represents  the  expected  severity  of  UH-60  losses  in  100,000  flying  hours 
if  no  controls  are  implemented.  The  simulation  is  then  run  again  with  the  control  effects 
included.  The  difference  between  the  base  case  severity  and  a  score  with  a  control 
implemented  is  the  control  performance  score. 

To  simulate  the  effect  of  implementing  a  control,  the  same  empirical  data  set  is 
transformed  based  on  the  hazard  and  control  assessments.  The  adjustment  percentage  is 
determined  by  calculating  the  casualty,  cost,  and  frequency  reduction  totals.  Each  control 
has  a  list  of  hazards  it  affects  along  with  the  estimated  percent  effectiveness.  Each 
accident  has  a  list  of  percent  contribution  of  each  hazard.  The  ASIST  casualty,  cost,  and 
frequency  assessments  are  applied  to  the  various  evaluation  measures.  It  would  be  best  to 
individually  assess  the  hazard  contribution  for  each  evaluation  measure  in  the  Severity  of 
Losses  Model,  which  would  require  convening  the  ASIST  experts  for  another  long  and 
expensive  session.  Since  the  hazard  assessments  developed  by  ASIST  were  readily 
available  and  had  consensus  approval  by  the  ASIST  members,  a  method  was  found  that 
would  allow  their  use  in  this  application. 

3.6.3.1  Hazard  Contribution  Assessments 

Hazard  contribution  to  casualty  assessments  are  applied  to  all  evaluation  measures 
in  the  casualties  branch,  as  well  as  the  unit  morale  evaluation  measures  under  unit 
readiness.  The  unit  morale  evaluation  measures  deal  with  fatalities  and  magnitude  of 
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injury  and  clearly  had  the  most  direct  ties  to  effectiveness  against  casualties  (see  Figure 
3-23). 

Hazard  contribution  to  cost  assessments  are  applied  to  total  costs  and  to  the 
evaluation  measures  in  equipment  availability  in  unit  readiness.  Equipment  availability, 
measured  in  terms  of  man-hours  to  repair,  is  a  component  of  the  total  accident  costs.  It  is 
therefore  assumed  the  hazards  that  are  effective  in  reducing  cost  are  equally  effective  in 
reducing  repair  effort. 

Hazard  contribution  to  frequency  is  used  for  training  execution  and  environmental 
damage.  Hazard  frequency  contribution  is  assessed  as  the  degree  (percent)  to  which  that 
hazard  is  responsible  for  causing  the  accident.  Since  training  execution  uses  number  of 
class  A-C  accidents,  it  was  logical  to  use  the  frequency  reduction  assessment  as  the  effect 
on  this  measure.  Hazard  contribution  assessments  for  environmental  damage  do  not 
currently  exist.  It  is  assumed  that  the  hazard  frequency  assessments,  at  this  point,  are 
adequate  to  approximate  the  infrequent  occurrence  of  environmental  damage  due  to  spills 
(Warren,  2000).  This  may  be  an  area  to  consider  in  the  further  development  of  the 
model.  The  frequency  assessments  are  also  used  to  simulate  the  reduction  in  the  number 
of  accidents. 
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Figure  3-23  Application  of  Hazard  Contribution  Assessments 

3.6.3.2  Accident  Data  Calculations 

When  an  accident  is  randomly  selected  in  the  simulation,  each  hazard  contribution 
assessment  is  combined  with  the  control  effectiveness  estimate  to  determine  the  casualty, 
cost,  and  frequency  reduction  amount. 

To  illustrate  this  process,  consider  the  following  hypothetical  accident,  hazards, 
and  controls: 


Case 

Table  3-3  Sample  Accident  Data 

Total  Costs  Fatalities  Repair  Hrs 

Oil 

19940205 

65,000 

0 

292 

1.5  Gallons 

Hazard 

Casualty 

Cost 

Frequency 

1 

1 

0.5 

0.5 

2 

0 

0.25 

0.5 

3 

0 

0.25 

0 

Now  assume  Control  1  is  40  percent  effective  on  Hazard  1,  Control  2  is  30 
percent  effective  on  Hazard  2,  and  60  percent  effective  on  Hazard  3.  If  Control  1  is 
implemented,  and  case  19940205  is  drawn  during  a  simulation  run,  the  amount  of 
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frequency  reduction  would  be  20  percent.  A  Bernoulli  trial  (success  or  failure)  is 
conducted  to  decide  if  the  accident  data  should  be  aggregated.  This  simulates  reduction 
of  the  accident  rate  due  to  implementing  controls.  If  the  Bernoulli  trial  is  a  success,  the 
accident  occurs,  and  the  accident  data  would  be  changed  to: 

Table  3-4  Control  Effectiveness 


Control 

Hazard 

1 

2 

3 

1 

0.40 

0 

0 

2 

0 

0.30 

0.60 

Because  Hazard  1  is  50  percent  responsible  for  the  costs,  and  Control  1  is  40 
percent  effective  on  Hazard  1,  this  amounts  to  a  20  percent  reduction  in  total  accident 
costs. 

new  total  accident  costs  =  [1  -  (0.5  *  0.4)]  *  65000 

Table  3-5  Control  1  Data  calculation 

Total  Costs  Fatalities  Repair  Hrs  Oil  Fuel 

52,000  0  2316  1.28  Gallons  0 

Repair  hours  are  reduced  in  the  same  fashion.  To  transform  the  amount  of  oil 
spilled,  using  the  frequency  assessment,  Hazard  1  is  50  percent  responsible.  Again,  this 
is  a  20  percent  in  reduction  in  oil  spilled.  In  the  Severity  of  Losses  Model,  the  new  oil 
amount  is  categorized  for  use  in  the  oil  severity  function. 

For  Control  2,  assuming  the  Bernoulli  trial  passes,  Hazards  2  and  3  are  affected 
and  each  contribute  25  percent  to  the  cost.  Given  that  Control  2  is  30  percent  effective  on 
Hazard  2,  and  60  percent  effective  on  Hazard  3,  this  amounts  to  a  22.5  percent  overall 
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reduction  in  total  costs.  Given  that  repair  hours  uses  the  same  hazard  cost  contribution 
assessment,  it  is  also  reduced  22.5  percent. 

new  total  costs  =  [1  -  (0.25  *0.3)  +  (0.25  *0.6)]*  65000 
Hazard  3  did  not  contribute  to  the  frequency  of  the  accident,  and  therefore  has  no 
effect  on  oil  spilled.  However,  because  of  the  contribution  of  hazard  2  to  the  reduction  of 
frequency,  the  overall  control  reduction  of  oil  spilled  is  15%. 
new  oil  spilled  =  [  1  -  (0.5  *0.3)]  *1.5 

Table  3-6  Control  2  Data  Transformation 

Total  Costs  Fatalities  Repair  Hrs  Oil _ Fuel _ 

50,375  0  226.3  1.2  Gallons  0 

3.6.4  Data  Aggregation 

After  the  accident  data  is  transformed  to  account  for  implementation  of  the 
control,  it  is  aggregated  with  data  from  the  other  randomly  selected  accidents.  For 
evaluation  measures  in  the  casualties  and  total  costs  branch  of  the  hierarchy,  the 
respective  data  is  summed  over  the  100,000  flying  hours.  The  severity  functions  assign  a 
severity  score  to  each  of  these  totals.  The  key  here  is  to  assess  the  weights,  making  sure 
the  key  experts  or  decision  makers  are  completely  aware  of  the  levels  of  aggregation  of 
the  measures  being  compared.  For  example,  training  execution  and  total  costs  are 
aggregated  at  different  levels.  Comparing  these  two  could  be  confusing  without  a 
detailed  description  of  the  differences. 
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3.6.4.1  Unit  Readiness  Data  Aggregation 

Each  severity  function  in  unit  readiness  is  focused  at  the  unit  level.  Individually, 
the  maximum  severity  for  a  unit  in  each  of  these  measures  is  one.  It  is  not  feasible, 
however,  for  every  unit  to  have  the  highest  level  of  a  measure  in  100,000  flying  hours. 
For  example,  if  there  are  15  accidents  in  100,000  flying  hours  and  the  maximum  in  the 
range  for  training  execution  represents  3  Class-A  accidents,  then  at  most  there  would 
only  be  5  out  of  the  26  units  with  this  maximum.  The  maximum  expected  severity 


for  all  units  combined  was  determined  through  Monte  Carlo  simulation.  The  average 
severity  for  the  units  in  each  run  is  normalized  to  the  maximum  expected  severity. 


Training  Execution  Severity  = 


- V  severitymu  ( training  _  execution ) 

mmumts~_ _ _ 

maximum  expected  severity  of  training  execution 


3.6.4.2  Environmental  Damage  Data  Aggregation 

Environmental  damage  assesses  the  severity  of  the  maximum  occurrence  of  a 
hazardous  fluid  spill  category  in  100,000  flying  hours.  In  the  accident  database,  spills 
are  documented  in  categories.  When  an  accident  is  selected  the  amount  of  spillage  is 
estimated  using  the  following  conversion: 


Table  3-7  Environmental  Categories 


Category 

Estimation 

0 

No  Spills 

0 

1 

0-1  Gallons 

.5  Gallons 

2 

1-2  Gallons 

1 .5  Gallons 

3 

2-10  Gallons 

6  Gallons 

4 

10-20  Gallons 

15  Gallons 

5 

>  20  Gallons 

30  Gallons 
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The  estimated  amount  of  spillage  is  then  reduced,  as  described  in  section  3.9.2,  to 
simulate  the  implementation  of  the  control.  The  reduced  amount  is  re-categorized  and 
compared  with  the  current  maximum  category  for  that  type  of  spill  in  the  simulation.  The 
maximum  categorical  occurrence  for  fuel,  hydraulic  fluid,  and  oil  is  the  level  used  in  the 
environmental  damage  severity  functions. 

3.7  Expected  Severity 

Having  developed  the  severity  functions,  determined  the  weights,  and  aggregated 
the  data,  determining  expected  severity  is  straightforward.  Using  the  additive  value 
function,  each  severity  score  is  multiplied  by  its  respective  weight  (Kirkwood,  1997:  72). 
The  sum  of  the  weighted  severity  scores  is  the  severity  for  UH-60  accidents  in  the 
100,000  flying  hours.  In  the  Monte  Carlo  simulation,  this  process  is  repeated  n=10  times 
in  each  of  the  B  replications,  creating  a  distribution  of  B  expected  severity  scores. 

3.7.1  Control  Performance  Scoring 

Initially,  the  simulation  is  run  with  no  controls  selected.  This  is  referred  to  as  the 
base  case.  The  base  case  represents  the  current  situation,  assuming  no  new  controls  have 
been  implemented.  The  simulation  is  then  run  incorporating  the  effectiveness  of  a  given 
control.  Control  performance  is  the  difference  between  the  base  case  expected  severity 
and  the  expected  severity  with  the  control  implemented.  Each  control  performance  score 
is  representative  of  the  expected  severity  reduction  resulting  from  implementing  that 
individual  control.  The  expected  effect  of  implementing  multiple  controls  is  not 
necessarily  additive.  The  assessed  percent  effectiveness  on  the  affected  hazards  could 
add  up  to  more  than  100  percent  when  implementing  more  than  one  control.  If  this  were 
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the  case,  the  expected  decrease  in  the  severity  of  losses  would  be  overestimated.  Because 
of  this  distinction,  individual  performance  scores  cannot  be  simply  added. 

To  evaluate  the  expected  decrease  in  severity  resulting  from  implementing  a 
“group”  or  portfolio  of  controls,  the  effectiveness  of  each  control  on  each  hazard  is 
combined,  up  to  a  maximum  of  100  percent.  Using  the  portfolio  effectiveness  in  place  of 
individual  control  effectiveness,  portfolio  performance  is  calculated  as  the  difference 
between  the  base  case  expected  severity  and  the  expected  severity  with  the  portfolio 
implemented.  For  example,  if  controls  in  Table  3-7  were  combined  into  a  portfolio,  the 
actual  effect  on  the  hazards  would  be  the  combined  effect,  not  the  additive  effect. 


Table  3-8  Sample  Portfolio  Effectiveness 


Control  No 

AVN-01 

AVN-02 

AVN-03 

AVN-04 

AVN-05 

H60-02 

H60-08 

H60-44 

H60-59 

25  °/t 

25% 

25  % 

5% 

5% 

5% 

5% 

5%o 

H60-C05 

60°/( 

30°/c 

60°/c 

i 

■■■■■■■ 

BiHB 

wmm 

Added  %  Effective 

225 °/ 

i  25°/ 

55% 

5% 

5% 

5% 

65% 

5% 

70% 

Combined  %  Effecti 

ve  100°/ 

i  25°A 

55% 

5% 

5% 

5% 

65% 

5 % 

70% 

3.8  Portfolio  Analysis 

Each  control  has  an  estimated  cost  and  estimated  effectiveness  percentages  on  its 
associated  hazards.  Each  portfolio  will  have  a  total  cost  and  its  combined  effectiveness 
on  hazards.  A  method  of  determining  which  controls  to  include  in  a  portfolio  was 
required.  One  way  of  modeling  such  problems  is  with  integer  programming  where  each 
control  is  judged  either  yes  or  no  for  inclusion  in  the  portfolio.  The  objective  function 
could  be  to  minimize  cost,  with  the  constraints  defined  as  the  achievement  of  50 
reduction  in  severity.  However,  in  order  to  demonstrate  cost  versus  effectiveness,  cost 
was  used  as  a  constraint.  The  objective  was  to  select  controls  that  have  the  highest 
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percentage  effect  on  the  most  hazards,  or  maximize  the  coverage  on  the  hazards.  Solving 
the  integer  program  at  various  cost  levels  generated  a  variety  of  portfolios  to  consider. 

This  integer  program  does  not  consider  the  severity  of  the  individual  hazards 
when  maximizing  coverage.  Thus,  there  is  no  guarantee  that  the  ‘optimal’  generated 
portfolio  will  achieve  the  best-simulated  performance.  However,  this  heuristic  is  very 
quick  and  coverage  should  correspond  nicely  with  total  control  effectiveness.  An  iterative 
process  is  used  to  generate  portfolios,  which  are  simply  starting  points  in  the  search  for 
the  best  performing  portfolio  for  specified  cost  threshold.  Each  iteration  of  generating 
maximal  coverage  portfolios  involves  adding  constraints  to  force  the  integer  program  to 
choose  a  different  mix  of  controls.  This  process  creates  a  diverse  group  of  portfolios 
whose  performance  is  then  estimated  using  the  Monte  Carlo  simulation  described  above. 

The  integer  program  is  implemented  in  Microsoft  Excel  using  Premium  Solver 
add-in  package  from  Front-line  Systems  (1999).  A  complete  description  of  the 
formulation  of  this  integer  program  is  included  in  Appendix  D. 

3.9  Summary 

The  Severity  of  Losses  Model,  which  integrates  doctrine,  expert  opinion,  and 
commander  preferences,  quantifies  the  severity  of  losses  resulting  from  100,000  UH-60 
Blackhawk  flying  hours.  A  zero/one  integer  program  generates  portfolios  by  selecting 
controls  that  maximize  hazard  coverage  for  a  given  maximum  cost.  The  Monte  Carlo 
simulation  using  the  bootstrap  method  simulates  the  effect  of  implementing  a  control  or 
portfolio  of  controls  by  using  the  ASIST  control  effectiveness  percentages  and  hazard 
contribution  assessments.  Lastly,  the  Severity  of  Losses  Model  quantifies  the  expected 
decrease  in  severity  of  losses  by  comparing  the  difference  before  (base  case)  and  after 
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implementing  controls.  The  framework  in  Figure  3-1  summarizes  the  methodology  used 
in  this  study.  Results  and  analysis  from  this  methodology  are  discussed  in  the  next 
chapter. 
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4  Results  and  Analysis 


4.1  Introduction 

This  chapter  contains  analysis  and  discussion  of  the  results  from  Severity  of 
Losses  Model  developed  in  Chapter  3.  The  first  section  compares  the  ten  most  severe 
UH-60  Blackhawk  hazards  identified  from  this  analysis  with  ASIST’s  ranking.  Next, 
control  performance  scores  and  portfolio  composition  and  scores  are  discussed.  After 
reviewing  all  portfolios,  six  portfolios  of  controls  that  meet  the  50  percent  reduction  goal 
are  further  analyzed.  Some  final  thoughts  on  the  findings  of  this  project  conclude  the 
chapter. 

4.2  Monte  Carlo  Replication  s 

Prior  to  evaluating  hazards,  controls,  or  portfolios  of  controls,  the  number  of 
replications  for  the  simulation  needed  to  be  determined.  Efron  and  Tibshirani  state  that 
200  replications  are  sufficient  for  estimating  the  mean,  but  generally  need  to  be  increased 
by  a  factor  of  ten  when  computing  bootstrap  confidence  intervals  (Efron  and  Tibshirani, 
1993:15). 

The  confidence  interval  half-length  is  used  to  determine  the  number  of 
replications  necessary  to  generate  confidence  intervals  with  specified  precision  e.  The 
number  of  replications  R  necessary  for  a  95  percent  confidence  interval  with  precision  of 
0.01  is  the  smallest  integer  such  that  R>Ro  and  satisfies  the  following  equation,  where  Ro 
replications  were  used  to  obtain  the  sample  variance,  So 2  (Banks,  et  al.,  1999:  449) 

f  taH.R-lSo'' 

R  >  - 

V  s 
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Using  this  method,  with  an  initial  sample  size  of  20,  the  required  number  of 
replications  was  41.  Therefore,  the  number  of  replications  used  in  this  study  was  410 
(based  on  the  factor  of  ten  rule  of  thumb)  (Efron  and  Tibshirani,  1993:15). 

4.3  Hazard  Severity 

Determining  which  hazards  are  the  most  severe  is  useful  when  considering 
control  performance.  One  way  of  defining  hazard  severity  is  the  amount  of  contribution 
from  the  hazard  to  the  overall  severity  of  losses.  Consider  a  hypothetical  control  that 
eliminates  100  percent  of  a  hazard.  If  this  hypothetical  control  were  implemented,  the 
decrease  in  the  overall  expected  severity  of  losses  in  100,000  flying  hours  would 
represent  that  hazard’s  severity. 

Table  4-1  contains  a  comparison  of  the  top  ranking  hazards  from  the  Severity  of 
Losses  Model  and  from  ASIST.  The  table  is  sorted  by  severity,  with  AVN-04  ranked  as 
the  most  severe  hazard.  While  there  is  some  consistency  in  the  makeup  of  the  two 
rankings,  the  differences  are  worth  noting.  The  major  contributing  factor  to  the 
difference  in  the  two  rankings  is  due  to  the  accounting  of  the  frequency  of  the  hazards. 
ASIST  includes  the  number  of  occurrences  of  a  hazard  as  a  variable  in  its  objective 
function,  while  the  Severity  of  Losses  Model  uses  the  hazard  frequency  contribution  to 
accidents  as  a  probability  for  that  accident  occurring.  For  example,  H60-04  is  100 
percent  accountable  for  the  frequency  of  every  accident  in  which  it  occurs.  Its  severity 
ranking  is  high  because  all  H60-04  accidents  are  omitted  when  calculating  the  severity  of 
losses.  Assuming  the  entire  hazard  is  eliminated,  each  time  a  H60-04  accident  is  drawn 
its  associated  accident  data  is  not  aggregated. 
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Table  4-1  Hazard  Ranking  Comparison 


Hazard 

Hazard  Definition 

Severity 

Rank 

ASIST 

Rank 

AVN-04  (13) 

Commanders  selectively  enforce  standards 

(published  standard  is  not  the  same  as  the  accepted  standard) 

1 

1 

H60-10  (9) 

Performing  maintenance  procedures  incorrectly  can  result  in  flight 
hazards. 

2 

6 

H60-04  (12) 

Aft  cyclic  input  or  main  rotor  blade  flex  may  cause  main  rotor  blade 
contact  with  aircraft  components  when  landing. 

3 

11 

AVN-02  (13) 

Commander  or  unit  lack  experience,  wisdom,  or  seasoned  leadership  to 
apply  risk  management 

4 

3 

H60-58  (1) 

Aircraft  operations  in  close  proximity  under  high  workload  conditions 
may  result  in  loss  of  situational  awareness  and  a  multi-aircraft  collision. 

5 

5 

H60-18  (9) 

Materiel  failure  of  engine  components  may  result  in  a  loss  of  engine 
power 

6 

2 

H60-08  (2) 

Attempting  maneuvers  (due  to  a  lack  of  understanding)  which  require 
more  power  or  lift  available  will  result  in  loss  of  control  of  the  aircraft 

7 

7 

AVN-01  (8) 

Commander  lacks  factual  and  timely  information  for  managing  high  risk 
behavior 

8 

9 

H60-03  (6) 

Aircraft  operations  in  close  proximity  to  unimproved  surfaces  may  result 
in  degraded  visual  environment  leading  to  loss  of  situational  awareness 
and  damage  to  aircraft 

9 

12 

H60-09  (2) 

Crew  may  be  unable  to  identify  a  failed  engine  resulting  in  shutdown  of  a 
properly  operating  engine  in  flight. 

10 

13 

H60-06  (22) 

Loss  of  situational  awareness  (as  a  result  of  distance  estimation,  varying 
workload,  environmental,  and  visual  issues)  while  maneuvering  in  close 
proximity  to  trees  or  objects  may  result  in  the  aircraft  striking  the  trees  or 
objects 

11 

10 

H60-23  (1) 

Occupants  may  be  exposed  to  post-crash  fire  by  use  of  ferry  tanks 
(ERFS)  with  less  crashworthiness  than  main  fuel  tanks. 

12 

4 

H60-46  (1) 

Miscommunication  between  aircrew  and  jumpmaster  (heilocast)  may 
cause  early  exit  from  aircraft  resulting  in  injury  to  jumpers. 

13 

16 

H60-24  (1) 

External  fuel  tank  structural  integrity  may  be  exceeded  by  crash  forces  in 
otherwise  survivable  crashes. 

14 

20 

H60-01  (3) 

The  aircrew’s  ability  to  visually  acquire  and  avoid  all  wires  may  be 
reduced  when  crossing  known  wires  between  the  poles  under  a  degraded 
visual  environment,  which  may  result  in  a  wire  strike 

15 

15 

H60-55  (1) 

Personnel  or  equipment  may  be  struck  by  main  rotor  blades  if  adequate 
ground  clearance  does  not  exist  during  hot  refuel. 

16 

8 

H60-16  (3) 

Failure  of  the  tail  strut 

17 

22 

H60-05  (8) 

Loss  of  situational  awareness  (as  a  result  of  distance  estimation,  fixation 
and  scanning  issues)  while  ground  taxiing  in  close  proximity  to  objects 
may  result  in  blade  strikes. 

18 

27 

H60-49  (1) 

Aircraft  operations  in  degraded  visual  environment  (night  aided  over 
water)  may  result  in  loss  of  situational  awareness  resulting  in  the  aircraft 
striking  an  obiect 

19 

19 

(number  of  accidents  in  which  hazard  contributed) 
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4.4  Control  Performance 


Control  performance  is  defined  as  the  expected  decrease  in  severity  of  losses,  in 
100,000  flying  hours,  resulting  from  the  implementation  of  a  control.  As  described  in 
Chapter  3,  each  control  may  affect  a  number  of  hazards  contributing  to  accidents,  as 
assessed  by  ASIST.  Using  the  control  effectiveness  and  hazard  contribution  assessments, 
the  performance  of  the  control  is  approximated.  The  definitions  of  the  controls  are  in 
Appendix  B. 

Table  4-2  contains  the  top  15  individual  controls  ranked  by  expected  decrease  in 
severity.  Controls  C34  and  C47  have  moved  up  dramatically  in  ranking  compared  with 
the  original  ASIST  Ranking. 


Table  4-2  Top 

15  Contra 

Performance  Results 

Control 

Investment 

Costs 

(SMillions) 

Expected 

Severity 

Decrease 

Percent 

Expected 

Decrease 

Severity  of 

Losses 

Rank 

ASIST 

Rank 

H60-C34 

200 

0.0230 

17.99% 

1 

8 

19 

0.0187 

2 

1 

3 

5 

H60-C44 

19 

0.0156 

12.21% 

4 

10 

H60-C11 

200 

0.0136 

10.62% 

5 

2 

H60-C47 

19 

0.0128 

9.99% 

6 

24 

H60-C07 

19 

0.0126 

9.83% 

7 

3 

H60-C01 

19 

0.0113 

8.83% 

8 

14 

H60-C42 

19 

0.0109 

8.55% 

9 

11 

7.81% 

10 

6 

19 

jB| 

7.23% 

11 

13 

H60-C43 

19 

0.0084 

6.58% 

12 

12 

H60-C17 

19 

0.0080 

6.23% 

13 

17 

H60-C31 

200 

14 

19 

H60-C03 

19 

22 
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Major  contributing  factors  to  these  control  performance  scores  are  the  decreases 
in  casualties  and  unit  readiness.  Figure  4-1  shows  how  the  top  controls  break  out  into 


each  area.  Note  the  y-axis  is  only  shown  to  50  percent  to  increase  readability  and 
indicate  the  Army’s  goal  of  50  percent  reduction. 
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Figure  4-1  Control  Performance  Breakdown 
While  it  appears  as  if  environmental  damage  is  not  included  in  the  Figure  4-1, 

upon  closer  inspection,  the  dark  line  at  the  top  of  some  bars  represents  the  contribution 

from  environmental  damage  to  reduction  in  severity.  This  small  effect  is  due  to  the  low 

weight  assigned  to  that  branch  of  the  hierarchy.  While  environmental  damage  would 

decrease  after  implementing  controls,  the  resulting  decrease  in  severity  is  very  small. 

No  single  control  can  achieve  the  Army’s  goal  of  50  percent  reduction  of  losses, 

therefore  portfolios  of  controls  must  be  considered  if  the  goal  is  to  be  achieved. 
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Figure  4-2  Expected  Reduction  in  Accident  Losses 
In  considering  the  control  performance  results,  the  question  of  what  severity 

decrease  means  in  terms  of  fatalities,  accident  costs  and  accident  rate  often  arises.  The 

expected  decrease  in  each  of  these  areas  for  the  top  controls  is  shown  in  Figure  4-2.  The 

black  bar  represents  the  expected  decrease  in  fatalities  in  100,000  UH-60  flying  hours 

after  implementing  the  control.  It  takes  approximately  six  months  to  accumulate  100,000 

flying  hours.  Over  the  past  five  years,  the  Army  has  averaged  8.9  Class  A-C  accidents,  2 

fatalities,  and  $85  million  lost  per  100,000  flying  hours.  Using  these  averages,  a  10 

percent  expected  decrease  in  these  categories  would  be  about  1  or  2  fewer  accidents  per 

year.  Fatalities  would  be  reduced  to  1.8  per  100,000  flying  hours,  or  approximately  1 

less  fatality  in  a  two  year  timespan.  Annual  accident  cost  savings  would  be 

approximately  $17  million.  It  appears  that  if  the  50  percent  reduction  in  severity  of 
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losses  goal  is  reached,  so  too  will  a  50  percent  reduction  in  fatalities  and  total  accident 
costs.  Again,  the  y-axis  is  shown  to  50  percent  to  increase  readability. 

4.5  Portfolio  Analysis 

Using  the  mathematical  program  developed  and  described  in  Chapter  3,  portfolios 
of  controls  were  developed.  A  frontier  of  portfolios  was  generated  by  using  the  program 
to  develop  a  varied  number  of  portfolios  at  $20  million  increments.  The  purpose  is  to 
give  a  graphical  representation  of  the  best  ‘bang  for  the  buck’.  $20  million  increments 
were  chosen  to  allow  for  the  addition  of  a  single  low  cost  control  (approximately  $19 
million)  at  each  higher  level.  With  more  accurate  cost  estimates,  smaller  increments 
could  be  investigated. 

Figure  4-3  contains  a  sample  of  portfolios  along  with  a  description  of  the  controls 
and  costs  included.  For  example,  Portfolio  of  Controls  138  (Portfolio  138)  costs  an 
estimate  $60  million,  and  contains  UH-60  controls  2,  7,  9, 18,  19,  22, 23,  38,  and  44. 
Using  the  Severity  of  Losses  Model,  the  expected  decrease  in  severity  having 
implemented  all  controls  in  the  portfolio  was  calculated. 

4.5.1  Portfolio  Performance 

Portfolios  are  evaluated  in  the  same  manner  as  individual  controls  by 
combining  the  assessed  effectiveness  of  each  control  in  a  portfolio.  The  maximum 
reduction  amount  for  a  portfolio  on  a  single  hazard  was  limited  to  100  percent.  The 
mathematical  program  generated  portfolios  by  selecting  controls  to  maximize  coverage 
on  the  hazards.  Other  portfolios  were  generated  by  manually  selecting  a  mix  of  controls 
based  on  a  benefit/cost  ratio.  A  complete  listing  of  the  portfolios  is  included  in  Appendix 
E,  ranked  by  expected  reduction  in  severity. 
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Control  Number 


Figure  4-3  Portfolio  Composition  Matrix 


The  following  chart  plots  portfolio  performance  versus  cost.  The  highlighted 
point  at  a  cost  of  approximately  $1180  million  represents  a  portfolio  containing  all 
controls  necessary  to  maximize  hazard  coverage,  with  no  limit  on  costs. 


Investment  Costs  ($Million) 

Figure  4-4  Portfolio  Performance  vs.  Cost 
Figure  4-5  is  a  similar  look  at  portfolio  performance,  with  the  focus  limited  to 

$200  million.  The  dark  curve  on  the  upper  portion  of  the  graph  represents  the  simple,  but 

mathematically  incorrect,  performance  sum  of  the  individual  controls  in  the  portfolio. 

Since  the  effect  on  a  hazard  is  limited  to  100  percent,  this  sum  could  be  a  false 

representation  of  what  is  actually  calculated  when  the  controls  are  combined. 

Additionally,  all  interactions  between  controls  are  also  ignored.  The  thinner  line  cutting 

across  the  middle  of  the  chart  shows  the  calculated  best  available  frontier.  The  various 
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points  on  the  graph  represent  calculated  portfolio  performance  scores. 


♦  Scored  Portfolios 
—  "False"  Frontier 
- Portfolio  Frontier 


Figure  4-5  Portfolio  Performance  Frontier 

4.5.2  Safety  Goals  Results 

A  stated  objective  of  this  project  is  to  identify  portfolios  of  controls  that  would 
help  reduce  losses  by  50  percent  over  the  next  ten  years  at  minimal  investment  costs. 
Table  4-3  contains  a  selection  of  portfolios  each  with  an  expected  decrease  in  severity  of 
losses  of  50  percent.  While  the  graph  in  Figure  4-5  portrays  the  50  percent  threshold 
being  reached  at  just  above  $60  million,  this  can  be  misleading.  A  portfolio  containing 
three  low  cost  controls  at  $19  million  each  as  well  as  all  controls  that  cost  less  than  $1 
million  each  would  total  $60.6  million.  Every  portfolio  evaluated  with  this  level  of 
investment  cost  fell  below  the  50  percent  expected  reduction  of  severity  level.  Adding 
one  more  control  to  the  portfolio  incurs  an  additional  cost  of  $19  million.  Figure  4-6 
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displays  the  control  composition  of  each  portfolio  that  met  the  reduction  threshold  for  the 
lowest  investment  costs. 


Portfolio  Performance 
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■  Casualties  ■  Unit  Readiness  m  Total  Costs  □  Environmental  Damage 


Figure  4-6  Portfolio  Performance  Breakdown  -  50%  Reduction  of  Losses 


The  primary  differences  between  the  portfolios  shown  above  are  a  change  in  one 
$19  million  control  and  one  or  two  less  expensive  controls.  For  example,  the  difference 
between  Portfolio  161  and  Portfolio  162  is  that  control  9  is  swapped  with  control  13,  and 
controls  23  and  33  are  added,  which  accounts  for  the  million  dollar  difference  in  portfolio 
costs. 

Considering  the  composition  of  the  portfolios  and  their  closeness  in  performance 
scores,  it  follows  that  the  breakdown  of  the  scores  also  show  similarities  (see  Figure  4-7). 
The  breakdown  of  portfolio  performance  scores  parallels  the  breakdown  of  individual 
control  performance  scores  with  casualties  and  unit  readiness  as  the  major  contributors. 


75 


Table  4-3  Portfolios  -  50%  Reduction  of  Losses 
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The  expected  decreases  in  terms  of  fatalities,  costs,  and  accident  rate  for  the  top 
controls  are  shown  in  Figure  4-8.  As  with  the  individual  controls,  the  major  decreases 
are  in  fatalities  and  total  costs.  Given  that  the  expected  decrease  in  overall  severity  is  so 
close  for  each  portfolio  this  breakdown  into  fatalities,  costs,  and  accident  rate  may 
provide  additional  insight  to  the  decision  maker. 
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Figure  4-8  Portfolio  Expected  Reduction  in  Accident  Losses 

4.6  Unit  Readiness  Weight  Sensitivity 

A  major  uncertainty  in  this  project  is  the  composition  of  the  unit  readiness 
severity  functions,  as  stated  in  the  development  phase  of  the  severity  hierarchy. 
Sensitivity  analysis  was  performed  by  varying  the  weight  of  unit  readiness  to  see  what 
impact  it  would  have  on  the  results.  The  current  weight  of  unit  readiness  is  0.378.  The 
weight  was  varied  from  0.25  to  0.45  with  the  results  shown  in  Figure  4-9. 
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While  it  would  be  difficult  to  make  out  the  rank  order  differences  graphically  in  a 
scaled  graph,  the  rank  order  of  the  portfolios  does  indeed  change,  as  shown  in  Figure  4-9. 
The  change  in  rank  order  of  the  top  three  portfolios,  however,  should  not  be  of  statistical 
concern.  Using  the  Mann- Whitney  U test  with  an  a  of  0.05  to  compare  the  two  means,  it 
is  shown  that,  with  the  current  weight  of  0.378,  there  is  no  statistical  difference  between 
the  average  decrease  in  severity  of  the  first  and  third  ranked  portfolios  (see  Appendix  F). 
Having  no  statistical  difference  between  the  averages  implies  mathematical  indifference 
between  performance  scores  of  the  two  portfolios.  With  the  weight  below  0.378,  the 
same  three  portfolios  remain  at  the  top  with  the  only  statistical  difference  among  them 
occurring  when  the  weight  equals  0.35.  In  this  case,  the  top  portfolio,  Portfolio  163,  and 
the  third  ranked  portfolio,  Portfolio  177,  is  statistically  different.  Above  the  current 
weight,  the  same  three  portfolios  remain  at  the  top,  with  a  difference  in  the  means  of  the 
first  and  third  ranked  portfolios  when  the  weight  is  0.45 


Unit  Readiness  Weight  Sensitivity 


Weight  of  Unit  Readiness 


-■-PCI  62 
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Figure  4-9  Unit  Readiness  Sensitivity  -  Rank  Order 
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Sensitivity  analysis  results  show  that,  for  the  current  group  of  portfolios,  varying 
the  weight  of  unit  readiness  from  0.25  to  0.45  does  change  the  ranking  of  the  top  two 
portfolios.  Statistical  testing  shows  this  change  is  not  significant  at  the  0.10  level. 

Table  4-4  Unit  Readiness  Sensitivity  -  Portfolios  Expected  Decrease  in  Severity 

Unit  Severity  Weight 


0.25 

0.30 

0.35 

(Original  Weight) 
0.378 

0.40 

0.45 

Portfolio  162 

61.4% 

58.4% 

54.3% 

52.2% 

51.3% 

47.5% 

Portfolio  163 

62.5% 

58.5% 

54.9% 

53.0% 

50.4% 

45.8% 

Portfolio  177 

61.3% 

57.8% 

52.6% 

53.1% 

51.4% 

47.2% 

Portfolio  179 

58.5% 

54.7% 

51.2% 

49.0% 

47.6% 

45.4% 

Portfolio  180 

61.1% 

57.1% 

52.2% 

50.5% 

48.4% 

44.4% 

Portfolio  181 

60.1% 

55.6% 

52.5% 

51.0% 

48.9% 

45.6% 

4.7  Summary 

Using  the  Severity  of  Losses  Model,  the  severity  of  hazards,  and  performance  of 
controls  and  portfolios  of  controls  has  been  quantified.  This  chapter  presented  the  results 
from  the  model  and  a  one-way  sensitivity  analysis  on  the  weight  of  the  major  uncertain 
area  of  the  model. 

The  chapter  also  focused  on  a  group  of  portfolios  that  achieved  the  50  percent 
reduction  of  losses  threshold  while  minimizing  investment  costs.  These  six  portfolios 
were  very  similar  in  composition;  each  containing  four  low  cost  and  five  to  seven  very 
low  cost  controls.  Of  these  six,  Portfolio  162  and  Portfolio  163  remained  the  top  two 
when  the  weight  of  unit  readiness  stayed  between  0.25  and  0.45. 

Generally  speaking,  there  are  a  number  of  different  portfolios  of  controls  that  total 
about  $80  million  in  investment  costs  and  will  meet  the  50  percent  expected  reduction  in 
severity  goal.  Reaching  the  75  percent  expected  decrease  level  would  require  a  total  of 
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about  $180  million,  and  attaining  an  85  percent  expected  reduction,  which  would  require 
selecting  all  possible  contributing  controls,  would  be  about  $1.2  billion. 
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5  Conclusions  and  Recomm  endations 


5.1  Conclusions 

Over  the  past  few  years  the  Army  aviation  accident  rate  has  been  on  the  rise. 
ASIST  was  chartered  to  reverse  this  trend  with  the  overall  goal  of  reducing  the  losses  by 
50  percent,  interpreted  as  reducing  the  number  of  fatal  and  disabling  injuries,  accident 
costs,  and  the  Class  A-C  accident  rate  by  50  percent  over  the  next  ten  years.  Focusing  on 
the  UH-60  Blackhawk  initially,  ASIST  created  a  hazard  taxonomy  and  a  list  of  controls 
designed  to  reduce  those  hazards.  This  taxonomy  has  now  been  completed  for  the  active 
fleet  of  Army  air  vehicles.  The  methodology  developed  and  demonstrated  in  this  project 
takes  full  advantage  of  ASIST’s  efforts  by  building  a  severity  hierarchy,  appropriate 
measures  to  quantify  severity,  and  a  method  for  aggregating  preferences  of  key  decision 
makers.  This  effort  incorporated  the  hazard  taxonomy  into  a  Monte  Carlo  simulation. 
The  Severity  of  Losses  Model  quantifies  the  effect  of  implementing  a  control  or  portfolio 
of  controls  and  calculates  the  expected  reduction  in  severity  of  losses  due  to  accidents. 

The  Severity  of  Losses  Model  is  based  on  doctrine  and  incorporates  command 
preferences  as  well  as  expert  opinion.  The  mathematical  program,  while  simple  at  this 
stage,  focuses  the  simulation  effort  by  generating  initial  portfolios  that  maximize  the 
hazard  coverage  for  a  given  investment  cost  threshold.  Using  the  mathematical  program, 
the  Severity  of  Losses  Model,  and  Monte  Carlo  simulation,  portfolios  of  controls  that 
meet  the  goal  of  50  percent  reduction  in  losses  have  been  identified.  A  number  of 
portfolios  were  generated  at  various  cost  levels.  Figure  5-1  displays  the  results  for  the 


81 


Expected  Severity  Decrease 


top  three  performing  portfolios  and  results  for  the  three  lowest  cost  portfolios  that  meet 
the  50  percent  reduction  goal. 


Figure  5-1  Portfolio  Performance  Frontier 


Portfolio  Performance 
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$1140M 
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Figure  5-2  Portfolio  Performance 


5.2  Recommendations 


The  methodology  used  in  this  project  has  been  incorporated  into  ASIST.  ASIST 
personnel  were  involved  throughout  the  research  in  providing  data,  clarification,  and 
guidance.  Plans  are  underway  to  integrate  the  portfolio  evaluation  simulation  model  with 
the  entire  maturing  ASIST  database.  Opportunities  to  improve  the  model  exist,  but  have 
yet  to  be  explored.  Future  efforts  may  include  considering  some  of  the  following 
recommendations. 

5.2.1  Mathematical  Program 

The  mathematical  program  used  in  this  project  to  generate  portfolios  is  based  on 
the  assumption  that  the  more  hazard  coverage,  the  better  the  portfolio  will  perform.  The 
intent  is  not  to  guarantee  optimal  portfolios,  but  rather  to  find  a  good  starting  point,  or 
suggest  a  mix  of  controls  that  is  expected  to  perform  well. 

In  Chapter  Four,  hazard  severity  was  estimated  by  running  the  simulation  with 
100  percent  of  the  hazard  eliminated.  A  better  way  to  generate  initial  starting  portfolios 
could  be  to  maximize  the  total  amount  of  hazard  severity  reduction.  It  may  even  turn  out 
that  portfolios  generated  in  this  maimer  maintain  the  same  rank  order  as  the  simulated 
results.  Such  an  occurrence  is  referred  to  as  strategic  equivalence.  If  the  two  methods 
result  in  strategically  equivalent  portfolios,  the  simulation  would  only  be  required  to 
initially  determine  hazard  severity.  Highly  focused  portfolios  could  be  simulated  to 
determine  the  expected  severity  reduction.  This  type  of  improvement  will  be  needed  as 
the  fleet  of  aircraft  considered  and  the  scope  of  the  safety  effort  involved  with  it  grows. 


83 


Search  techniques  and  heuristics  can  be  used  to  improve  the  speed  and  results  of  the 
portfolio  search. 

5.2.2  Severity  of  Losses  Model 

The  Severity  of  Losses  Model  developed  in  this  project  is  a  continuation  of  the 
efforts  of  Captain  Brian  Sperling.  As  the  methodology  evolves,  so  too  should  the 
Severity  of  Losses  Model. 

The  unit  readiness  branch  of  the  hierarchy  is  the  prime  candidate  for  further 
development.  LTC  Semmens  stressed  the  need  to  interview  as  many  battalion 
commanders  as  possible  to  accurately  capture  the  impact  on  unit  readiness.  A  leadership 
conference  is  held  annually  at  Ft.  Rucker  and  is  attended  by  many  battalion  commanders. 
Future  efforts  should  coordinate  with  this  conference  and  exploit  this  opportunity  to  meet 
with  the  attendees.  Another  vehicle  for  interviewing  many  commanders  is  at  the 
Command  and  General  Staff  College  at  Ft.  Leavenworth,  Kansas.  Improving  the 
accuracy  of  the  tradeoff  preferences  and  severity  functions  of  this  area,  which  is  very 
important  to  current  leadership,  will  improve  the  overall  quality  of  the  model. 
Additionally,  this  would  communicate  the  intent  and  capability  of  the  methodology,  thus 
improving  its  credibility  within  the  aviation  community. 

Another  opportunity  to  improve  upon  the  quality  of  the  model  would  be  to  elicit 
the  severity  functions  and  weights  from  an  even  higher  level  of  Army  Leadership.  The 
current  weighting  reflects  the  values  of  the  USASC  Deputy  Commander,  COL  Warren. 
Perhaps  the  Chief  of  Staff  of  the  Army  or  the  Secretary  of  the  Army’s  preferences  could 
be  elicited  to  accurately  represent  the  values  and  preferences  of  a  500,000  soldier 
organization. 
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The  final  comment  on  the  Severity  of  Losses  Model  has  to  do  with  the 
environmental  damage  aspects  of  the  hierarchy.  FM  100-14  clearly  states  that  damage  to 
the  environment  must  be  considered  when  evaluating  hazards.  FM  20-400  describes  the 
components  of  environmental  damage  (see  Figure  5-2).  Environmental  damage,  while 
considered,  is  not  highly  weighted  in  the  Severity  of  Losses  Model.  If  more  data  were 
collected,  the  actual  severity  of  environmental  damage  could  be  better  represented.  A 
more  developed  depiction  of  environmental  damage  may  result  in  higher  weighting.  The 
cost  of  environmental  cleanup  could  also  be  included  in  the  total  accident  costs. 


Oil  Oil 


Figure  5-3  Proposed  Environmental  Considerations 
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5.2.3  Control  Interactions  and  Dependencies 

The  Monte  Carlo  simulation  uses  the  control  effectiveness  assessments  to  adjust 
the  raw  accident  data,  thus  simulating  the  effect  of  a  control.  When  portfolios  are 
created,  control  effectiveness  assessments  for  a  hazard  are  assumed  to  be  additive,  up  to 
100  percent.  Depending  on  the  controls,  this  may  be  an  accurate  representation.  In 
reality,  the  actual  effects  may  not  truly  be  additive.  Individually,  two  controls  may  each 
reduce  a  given  hazard  by  50  percent.  Together,  the  assessed  effectiveness  may  turn  out 
to  be  75  percent.  Developing  the  effectiveness  relationships  will  involve  a  great  deal  of 
effort  on  the  part  of  the  ASIST  team  that  created  the  control  list  but  should  add  value  to 
the  overall  project. 

Along  the  same  line,  dependencies  between  controls  might  also  be  reinvestigated. 
This  model  includes  several  of  the  most  critical  dependencies  (see  Appendix  D),  but 
more  exist  and  should  be  incorporated. 

5.2.4  Monte  Carlo  Simulation 

The  model  is  built  in  an  Excel  Spreadsheet  and  uses  Crystal  Ball  to  run  the  Monte 
Carlo  Simulation.  Running  the  entire  process,  however,  is  interactive  and  somewhat 
time-consuming.  Future  efforts  may  include  a  purely  coded  program  that  would  be 
automated  for  the  user.  The  goal  of  this  research,  however,  was  to  find  a  methodology 
that  would  accomplish  the  objectives  set  forth  in  Chapter  One.  If  simulation  is  used  in 
future  efforts,  the  entire  process  could  be  automated. 

5.2.5  Future  Efforts 

This  project  focused  on  the  evaluation  of  portfolios  of  controls.  Future  efforts 
may  choose  to  consider  the  actual  budgeting  and  scheduling  for  implementation  of  the 
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selected  portfolio.  Incorporating  these  issues  into  the  model  will  produce  an  even  more 
representative  product  for  the  Army. 

5.3  Contributions 

The  combination  of  value  focused  thinking,  mathematical  programming,  and 
Monte  Carlo  simulation  has  been  well  received  by  the  safety  community  at  large.  As 
previously  mentioned,  ASIST  plans  on  incorporating  the  Severity  of  Losses  Model  into 
their  efforts. 

This  project  has  also  sparked  the  interest  of  members  of  the  Joint  Programs 
Opportunities  Board  (JPOB).  The  methodology  can  be  modified  to  work  with  any 
aircraft  system,  as  well  as  ground  systems  (more  lives  are  lost  on  the  ground  in  accidents 
than  are  lost  each  year  in  the  air).  The  eventual  use  of  the  Severity  of  Losses  Model  in 
the  joint  arena  is  uncertain,  but  the  potential  is  exciting.  The  methodology  will  be 
briefed  to  the  Joint  Aeronautical  Commanders  Group  in  April  2000  for  possible  use  by  all 
services  and/or  the  FAA  and  NASA. 
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Appendix  A  UH-60  Hazard  Taxonomy 


The  Aviation  Safety  Investment  Strategy  Team  (ASIST)  developed  the  hazards 
described  in  this  appendix.  This  taxonomy  is  the  result  of  an  exhaustive  investigation 
into  the  detailed  accident  reports  for  all  UH-60  Blackhawk  accidents  from  fiscal  years 
1994  through  1998.  Aviation  (AVN)  -code  hazards  cross  aviation  platforms,  while  H60- 
hazards  are  specific  to  the  Blackhawk. 

Hazard  Code  Hazard  Statement 

AVN-01  Commander  lacks  factual  and  timely  information  for  managing  high  risk 
behavior 

AVN-02  Commander  or  unit  lack  experience,  wisdom,  or  seasoned  leadership  to 
apply  risk  management 

AVN-03  Commander  lacks  understanding  of  the  available  range  of  controls  to 
manage  high-risk  behavior 

AVN-04  Commanders  selectively  enforce  standards  (published  standard  is  not  the 
same  as  the  accepted  standard) 

AVN-05  Commanders  are  unable  to  enforce  standards  because  of  reduction  in 
resources 

H60-01  The  aircrew’s  ability  to  visually  acquire  and  avoid  all  wires  may  be 

reduced  when  crossing  known  wires  between  the  poles  under  a  degraded 
visual  environment,  which  may  result  in  a  wire  strike 

H60-02  Unanticipated  fraying  and  breaking  of  hoist  cable  may  result  in  casualty 

H60-03  Aircraft  operations  in  close  proximity  to  unimproved  surfaces  may  result 
in  degraded  visual  environment  leading  to  loss  of  situational  awareness 
and  damage  to  aircraft 

H60-04  Aft  cyclic  input  or  main  rotor  blade  flex  may  cause  main  rotor  blade 
contact  with  aircraft  components  when  landing 
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Hazard  Code 


Hazard  Statement 


H60-05 

H60-06 

H60-07 

H60-08 

H60-09 

H60-10 

H60-11 

H60-12 

H60-13 

H60-14 

H60-15 

H60-16 

H60-17 

H60-18 

H60-19 


Loss  of  situational  awareness  (as  a  result  of  distance  estimation,  fixation 
and  scanning  issues)  while  ground  taxiing  in  close  proximity  to  objects 
may  result  in  blade  strikes 

Loss  of  situational  awareness  (as  a  result  of  distance  estimation,  varying 
workload,  environmental,  and  visual  issues)  while  maneuvering  in  close 
proximity  to  trees  or  objects  may  result  in  the  aircraft  striking  the  trees  or 
objects 

Loss  of  airframe  components  may  cause  damage  to  aircraft  during  flight 

Attempting  maneuvers  (due  to  a  lack  of  understanding)  which  require 
more  power  or  lift  available  will  result  in  loss  of  control  of  the  aircraft 

Crew  may  be  unable  to  identify  a  failed  engine  resulting  in  shutdown  of  a 
properly  operating  engine  in  flight 

Performing  maintenance  procedures  incorrectly  can  result  in  flight  hazards 

Unsecured  tools,  doors,  or  objects  on  or  around  the  aircraft  may  cause 
damage  during  run  up 

Unsecured  objects  within  the  aircraft  can  increase  personnel  injuries  in  the 
event  of  a  crash 

Untimely  emergency  response  to  hazardous  conditions  may  result  in 
increased  aircraft  damage 

Crew  will  unnecessarily  expose  personnel  to  injury  by  allowing  non- 
essential  personnel  on  board  aircraft 

Aircrew’s  ability  to  maintain  powered  flight  is  reduced  by  lack  of  fuel  to 
the  engine  caused  by  air  bubbles  in  the  fuel  system 

Failure  of  the  tail  strut 

Jettison  of  load  above  the  ground  may  result  in  damage  to  the  load 

Materiel  failure  of  engine  components  may  result  in  a  loss  of  engine 
power 

Crew  may  be  unable  to  properly  react  to  in  flight  emergencies  due  to  lack 
of  proficiency  with  NVGs 
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Hazard  Code  Hazard  Statement 

H60-20  Crew  may  voluntarily  elect  to  continue  flight  after  experiencing  an 
emergency  condition  which  could  result  in  injury  or  further  aircraft 
damage 

H60-21  Personnel  may  receive  unnecessarily  severe  injuries  when  personal 
protection  equipment  is  not  utilize 

H60-22  Crew  may  be  unable  to  perform  rescue/survival/injury  treatment  due  to 
inaccessibility  of  ALSE 

H60-23  Occupants  may  be  exposed  to  post-crash  fire  by  use  of  ferry  tanks  (ERFS) 
with  less  crashworthiness  than  main  fuel  tanks 

H60-24  External  fuel  tank  structural  integrity  may  be  exceeded  by  crash  forces  in 
otherwise  survivable  crashes 

H60-25  Crew  may  not  utilize  available  power  due  to  false  indications  from  MH- 
60K  voice  warning  system  which  does  not  automatically  reset  after  Main 
rotor  rpm  is  regained 

H60-26  Fire  extinguishing  system  failed  to  activate  in  a  crash  sequence  for 
unknown  reasons 

H60-27  Crew  may  misjudge  altitude  by  relying  on  the  MH-60K  aircraft  radar 
altimeter  information  displayed  on  the  multifunctional  display,  which, 
because  of  latency,  can  give  incorrect  information 

H60-28  Crew  may  experience  dual  engine  rollback  loss  of  power 

H60-29  Aircrew  seat  seatback  may  fail  prematurely  during  accident  sequence 

resulting  in  increased  severity  of  injuries 

H60-30  Crew  chief/gunner’s  ability  to  remain  securely  positioned  in  a  crash 
sequence  is  degraded  by  use  of  the  gunner’s  harness 

H60-3 1  Crew’s  ability  to  maintain  safe  body  position  in  a  crash  sequence  may  be 

degraded  by  use  of  a  single-mode  inertia  reel 

H60-32  When  flying  into  known  deteriorating  weather,  crew  may  lose  ability  to 
maintain  a  safe  flight  path  as  a  result  of  loss  of  situational  awareness 

H60-33  Fatigue  failure  of  tail  rotor  gearbox  bevel  gear  can  cause  loss  of  control 
resulting  in  forced  landing  and  aircraft  damage 
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Hazard  Code 
H60-34 

H60-35 

H60-36 

H60-37 

H60-38 

H60-39 

H60-40 

H60-41 

H60-42 

H60-43 

H60-44 

H60-45 

H60-46 


Hazard  Statement 

Foreign  object  blown  by  aircraft  rotor  down- wash  can  damage  aircraft 
components  causing  loss  of  aircraft  control  and  result  in  aircraft  crash 

Loss  of  situational  awareness  during  flight  in  degraded  visual  cues  may 
result  in  striking  of  objects  in  flight  path 

Crew  loss  of  situational  awareness  during  aircraft  operations  (left  engine 
cover  in  vicinity  of  main  rotor  system)  may  result  in  damage  to  the  aircraft 

Breakaway  fuel  fitting  failure  contributed  to  fire  and  resulting  aircraft 
damage.  Note:  Further  investigation  of  this  accident  by  aviation  systems 
needed  to  validate  the  hazard 

Flight  in  the  vicinity  of  lightning  may  result  in  aircraft  being  struck  by 
lightning 

Combining  training  with  mission  operations  (sling  load)  in  a  high 
workload  environment  can  cause  loss  of  situational  awareness  (divided 
attention)  resulting  in  aircraft/equipment  damage 

Use  of  an  aircraft  configured  with  full  ERFS  tanks  will  decrease  load¬ 
carrying  capability  in  sling  load  operations  and  may  result  in  a  low-rotor 
rpm  condition 

Inadvertent  release  (caused  by  trying  to  remove  excess  550  cord  from 
cargo  hook  system  manual  release  lever)  may  result  in  damage  to  the  load 

Unauthorized  testing  of  new  procedures  (sling  load  self  hook-up  method) 
may  result  in  aircraft/load  damage 

Combining  multiple  stressors  (fatigue,  OPTEMPO,  family  situation)  with 
mission  operations  (sling  load)  in  a  high  workload  environment  can  cause 
LOSA  (divided  attention)  resulting  in  aircraft/equipment  damage 

Mis-assessing  the  risk  of  an  identified  hazard  (ditch  in  landing  area)  may 
result  in  aircraft  damage 

Exiting  aircraft  during  jump  operations  at  high  altitude  and  airspeed  can 
result  in  injury  to  jumpers 

Miscommunication  between  aircrew  and  jumpmaster  (heilocast)  may 
cause  early  exit  from  aircraft  resulting  in  injury  to  jumpers 


Hazard  Code 
H60-47 

H60-48 

H60-49 

H60-50 

H60-51 

H60-52 

H60-53 

H60-54 

H60-55 

H60-56 

H60-57 

H60-58 

H60-59 

H60-60 

H60-61 

H60-62 


Hazard  Statement 

Hovering  aircraft  over  areas  with  loose  objects  can  cause  injury  to  ground 
personnel 

Hovering  aircraft  in  close  proximity  to  parked  aircraft  can  cause  aircraft 
damage 

Aircraft  operations  in  degraded  visual  environment  (night  aided  over 
water)  may  result  in  loss  of  situational  awareness  resulting  in  the  aircraft 
striking  an  object 

Aircraft  operations  in  a  degraded  visual  cue  (night  aided)  may  result  in 
loss  of  situational  awareness  resulting  in  the  aircraft  striking  an  object 

Lack  of  survival  radios 

Non- working  emergency  beacon 

No  overdue  aircraft  procedures 

No  risk  assessment  or  briefing 

Personnel  or  equipment  may  be  struck  by  main  rotor  blades  if  adequate 
ground  clearance  does  not  exist  during  hot  refuel 

Gusty  winds  may  affect  the  crew’s  ability  to  hold  the  aircraft  steady 
during  a  one  wheel  landing  to  a  slope  along  a  ridgeline 

Keeping  the  unisex  vent  valve  closed  during  refueling  may  result  in 
overpressurization  causing  damage  to  aircraft 

Aircraft  operations  in  close  proximity  under  high  workload  conditions 
may  result  in  loss  of  situational  awareness  and  a  multi-aircraft  collision 

Hazards  unknown 

The  unprepared  ELT  would  have  contributed  to  the  extent  of  the  injuries  if 
the  crash  had  taken  place  in  an  unpopulated  area 

Not  Enough  Information 

Tabled  for  lack  of  data 
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Appendix  B  Control  Listing 

The  controls  included  in  this  appendix  were  developed  by  ASIST  for  the  UH-60 


Blackhawk  aviation  program. 

H60-C01  Add  mandatory  scenario  training  in  simulator  (brownout)  to  include 
resourcing  of  TDY 

Estimated  Cost  (in  $Millions) 

$19.0 

Affected  Hazards 
H60-03 
H60-08 
H60-09 
H60-18 

H60-C02  Book  of  hazards  and  controls 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards 
AVN-01 
AVN-02 
AVN-03 
AVN-04 
AVN-05 
H60-02 
H60-08 
H60-44 

H60-C03  Change  ATM  to  establish  new  flying  hour  category  for  individual  task  flight 
training  hours  (not  collective  training) 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

H60-03  0.4 

H60-06  0.25 


Effectiveness 

0.25 

0.25 

0.25 

0.05 

0.05 

0.05 

0.05 

0.05 


Effectiveness 

0.1 

0.4 

0.4 

0.5 
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H60-C04  Develop  external  crashworthy  fuel  tanks  consider  suction  fuel  system 


H60-C05 


H60-C06 


H60-C07 


H60-C08 


Estimated  Cost  (in  SMillions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-23  0.75 

H60-24  0.75 

Establish  a  command  information  system  which  tracks  all  forms  of  high  risk 
behavior  and  marginal  performance 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

AVN-01  0.6 

AVN-03  0.3 

Establish  and  sustain  crewchief  s  "school  house"  training  program 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-06  0.35 

H60-58  0.35 


Establish  crew  coordination  sustainment  program 


Estimated  Cost  (in  $Millions) 
$19.00 


Affected  Hazards 

H60-03 

H60-06 

H60-32 

H60-58 


Effectiveness 

0.5 

0.5 

0.2 

0.4 


Establish  standards  for  and  resource  a  4th  crewmember  for  multi-ship 
operations.  Include  in 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-06  0.45 

H60-58  0.45 
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H60-C09  Expand  AQC  training  (emergency  procedures/emergency  diagnosis) 


Estimated  Cost  (in  $Millions) 
$19.00 


Affected  Hazards 

H60-08 

H60-09 

H60-18 

H60-58 


Effectiveness 

0.6 

0.6 

0.6 

0.5 


H60-C10  Implement  a  change  to  the  flight  control  system  to  improve  aircraft  stability 
and  control  in  low  speed  flight  (Attitude  Command  Attitude  Hold) 


Estimated  Cost  (in  SMillions) 
$90.00 


Affected  Hazards 

H60-03 

H60-04 

H60-06 

H60-58 


Effectiveness 

0.65 

0.5 

0.5 

0.2 


H60-C11  Increase  the  available  aircrew  experience 

Estimated  Cost  (in  $Millions) 

$200.00 


Affected  Hazards 

AVN-02 

H60-01 

H60-03 

H60-06 

H60-32 

H60-45 

H60-46 

H60-58 


Effectiveness 

0.05 

0.4 

0.2 

0.5 

0.5 

0.2 

0.2 

0.35 
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H60-C12  Modify  AR  95-XX  to  require  that  mission  planning  time  is  considered  as 
mandatory  topic  for  risk  determination  and  establish  risk  management 
standards  for  mission  planning  time 

Estimated  Cost  (in  $Millions) 

$0.05 

Affected  Hazards  Effectiveness 

AVN-01  0.01 

H60-01  0.01 

H60-03  0.01 

H60-08  0.01 

H60-C13  Provide  commanders  a  better  ability  for  selection,  mission  tailoring,  and 
balancing  of  resources  to  do  the  mission 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-02  0.4 

H60-08  0.5 

H60-32  0.5 

H60-58  0.4 

H60-C14  Develop  a  wire  strike  protection  system  that  covers  more  of  the  aircraft 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-01  0.15 

H60-C15  Enhance  NVG  field  of  view 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-06  0.25 

H60-C16  Develop  and  field  a  proximity  warning  system  (Virtual  Rotor  Disk) 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-06  0.7 
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H60-C17  Fund  and  install  flight  data  recorder  (FDR) 


Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-59  0.7 

H60-C18  Increase  command  emphasis  (Advance  Course)  on  safety  incentives 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-03  0.15 

H60-C19  Modify  AR  95-3  or  TC  1-210  to  require  2  hours  annually  of  actual 
instruments  for  each  PIC 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

H60-32  0.7 

H60-C20  Develop  and  field  an  adjustable  proximity  warning  sy stem/ collision 
avoidance. 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-58  0.7 

H60-C21  Develop  a  terrain  following  /  terrain  avoidance  radar 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-32  0.8 
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H60-C22  Develop  standardized  training  support  package  for  use  at  unit  level  targeted 
on  ERFS  operations  to  include  simulator  scenario  training,  jettison  stores,  a/c 
performance  characteristics 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

H60-23  0.5 

H60-C23  Manual  changes  to  describe  handling  characteristics 

Estimated  Cost  (in  SMillions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-08  0.05 

H60-C24  Develop  and  install  new  Night  Vision  Systems 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-06  0.5 

H60-C25  Improve  aircraft  controllability  with  tanks  installed  (pitch  bias  actuator, 
digital  stabilator  amp) 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-08  0.1 

H60-C26  Improve  IFR/IMC  infrastructure  in  selected  parts  of  the  world  (ABSO  define) 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-32  0.02 

H60-C27  Wire  detection  system  using  laser  radar  or  HF  radar  technology 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-01  0.6 
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H60-C28  Resource  aviation  maintenance  IAW  reference  XYX  to  match  requirements 
of  complex  aircraft,  (link  to  USAALS  needed  to  consider  dedicated  crews  to 
aircraft) 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-10  0.7 

H60-C29  Full  authority  DEC  automatically  causes  engine  shutdown 

Estimated  Cost  (in  SMillions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-09  0.9 

H60-C30  Improve  engine  diagnostics  and  improve  cueing  of  correlation  of  PCL  handle 
to  engine 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-09  0.7 

H60-C31  Accelerate  addressing  materiel  failures 

Estimated  Cost  (in  $Millions) 

$200.00 

Affected  Hazards  Effectiveness 

H60-18  0.9 

H60-C32  Improve  crew's  ability  to  for  premission  planning  by  implementing  electronic 
data  management  from  Air  Warrior 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-08  0.1 
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H60-C33  Enforce  rules  through  leadership  commitment 

Estimated  Cost  (in  SMillions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-08  0.05 

H60-C34  Digital  Source  Collector  (DSC)  and  envelope  cueing  (exceedences)  and 
notice  to  pilot  of  exceedences/crew  monitor 

Estimated  Cost  (in  SMillions) 

$200.00 

Affected  Hazards 
AVN-01 
H60-08 
H60-59 

H60-C35  Increase  minimum  distance  between  aircraft  for  multiship  operations 

Estimated  Cost  (in  SMillions) 

$0.05 

Affected  Hazards  Effectiveness 

H60-58  0.01 

H60-C36  Modify  manual  to  establish  method  of  calculating  lateral  CG 

Estimated  Cost  (in  SMillions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-08  0.15 

H60-C37  Establish  model  RM  training  program,  starting  with  DAIG  evaluation  of  all 
institutional  training  schools  for  integration  of  risk  management  training  into 
curriculum 

Estimated  Cost  (in  SMillions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-02  0.5 


Effectiveness 

0.7 

0.6 

0.7 
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H60-C38  Establish  or  enforce  selection  criteria  for  advanced  aviator  training 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-04  0.1 

H60-C40  Develop  a  standardized  methodology  for  conducting  mission  risk 

assessments  with  the  objective  for  identification  of  all  hazards  and  associated 
controls  to  be  presented  in  the  mission  brief 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-02  0.6 

H60-C41  Increase  the  system  specific  instruction  in  the  UH-60  IP  course.  Emphasize 
what  system  specific  instruction  should  be  imparted  by  IP's  to  other  aviators 
during  subsequent  assignments 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-01  0.7 

H60-C42  Expand  leader  development  training  to  emphasize  enforcement  of  aviation 
maintenance  and  operations  standards  (integration  into  advanced  course,  an 
exportable  training  package,  and  modifying  BOC  and  AOC) 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-04  0.4 

H60-C43  Integrate  risk  management  training  into  aviation  officer/WO/NCO 

development  programs  (use  accident  experience  as  part  of  the  training) 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

AVN-02  0.65 
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H60-C44  Install  Flight  Data  Recorders  (FDRs)  and  develop  procedures  for  use  of  FDR 
data  by  commanders  for  aircrew  training 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

AVN-01  0.7 

H60-59  0.7 

H60-C45  Evaluate  H-60  maintenance  force  structure. 

Estimated  Cost  (in  $Millions) 

$0.50 

Affected  Hazards  Effectiveness 

AVN-05  0.0001 

H60-C46  Investigate  improving  H-60  handling  qualities  by  implementing  strakes  to 
improve  airflow  over  the  tailboom 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-03  0.1 

H60-C47  Develop  a  smooth  deflection  device  on  top  of  ALQ-144 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-04  0.9 

H60-C48  Relocate  /redesign  ALQ- 1 44  (substitute  ATIRCMS) 

Estimated  Cost  (in  $Millions) 

$90.00 

Affected  Hazards  Effectiveness 

H60-04  0.5 
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H60-C49  Develop  and  distribute  an  exportable  training  package  for  unit  level  Aviation 
refuelers  and  mandate  training  requirement  prior  to  assignment  as  aviation 
refueler 

Estimated  Cost  (in  $Millions) 

$19.00 

Affected  Hazards  Effectiveness 

H60-55  0.4 
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Appendix  C  -  Severity  M  o  nte  Carlo  Simulation  Description 


This  appendix  provides  an  overall  description  of  the  Visual  Basic  program  used  to 
conduct  the  data  aggregation  and  severity  score  calculations.  A  description  of  the 
complete  process  to  evaluating  a  portfolio  of  controls  is  included. 

The  Visual  Basic  code  is  included  in  Appendix  G. 

In  conducting  the  simulation,  the  first  function  called  is  ‘do  a  rep’.  This  function 
is  responsible  for  calculating  an  expected  severity  score  representing  n  samples  of 
severity  scores.  Each  sampled  severity  score  comes  from  the  function  ‘rand_draw\  This 
function  is  where  everything  takes  place.  The  function  call  is  as  follows: 

rand_draw(num_draws,  haznum,  percent_eff,  controlnum,  numberofhazards) 
where  num  draws  =  the  number  of  accidents  (randomly  generated)  for  that 
100,000  flying  hour  period 

haznum  =  the  list  of  hazards  affected  by  the  current  control  list 
percent  eff  =  the  ordered  list  of  combined  effectiveness  estimates 
control_num  =  the  unique  number  of  the  control  or  portfolio  -  used  for  tracking 
raw  and  severity  totals 

number  of  hazards  =  the  number  of  hazards  affected 
The  function  executes  as  follows: 

For  i  =  1  to  number  of  draws 
-  randomly  pick  an  accident 

randomly  assign  a  unit  to  the  accident 
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based  on  the  hazards  and  the  accident,  determine  the  amount  of  casualty,  cost, 
frequency  and  environmental  reduction  for  the  current  accident 
based  on  the  amount  of  frequency  reduction  determine  if  the  accident 
occurred 

happen  =  random  uniform  variable  (0,1) 

If  happen  >  frequency  reduction,  then  the  accident  happened  -  continue  with 
function.  If  not,  the  accident  didn’t  happen  -  go  to  the  next  draw  i 
if  the  accident  did  happen,  reduce  the  raw  data  and  aggregate  accordingly 
Unit  data  is  tracked  by  unit  in  arrays 

Environmental  data  is  tracked  as  the  maximum  occurrence  of  a  category  of 
spill 

After  all  of  the  draws,  the  aggregated  data  is  processed. 

-  Unit  readiness  data  is  sent  to  the  severity  functions  and  then  averaged  over 
all  the  units 

-  The  rest  of  the  data  is  sent  to  their  respective  severity  functions 
If  tracked,  raw  and  severity  totals  are  written 

Finally,  each  branch  is  summed  up  and  the  function  returns  the  severity  score 
representing  the  losses  from  accidents  occurring  in  100,000  UH-60  flying 
hours 
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The  process  of  running  the  simulation  can  be  broken  down  into  three  phases: 


1)  Pre-simulation  setup 

2)  Running  the  simulation 

3)  Post-simulation  information  gathering 

Pre-simulation  setup  is  fairly  straightforward  and  involves  verifying  the  data  in  the 
spreadsheets  are  correct  and  the  parameters  in  the  Visual  Basic  code  (see  Appendix  G) 
are  correct. 

1)  Pre-simulation  setup 

-  Verify  the  cells  referenced  by  the  Visual  Basic  code  are  correct.  This 
would  only  need  to  be  done  if  the  severity  functions  change  or  if  hazards 
are  added. 

-  Verify  the  weights  on  the  top  of  the  ‘Accident  Data’  worksheet  are  correct 

-  Verify  all  global  constants  in  the  VB  code(num_units,  num  reps,  etc) 

-  If  the  raw  and  severity  totals  are  going  to  be  tracked,  the  global  arrays 
must  not  be  commented  out  and  the  num  controls  needs  to  be  set  to  the 
number  of  controls  that  is  going  to  be  tracked 

2)  Running  the  simulation 

-  Using  Crystal  Ball,  set  the  run  preferences  to  the  desired  settings 

-  Using  Crystal  Ball,  declare  an  assumption  cell  -  the  cell  is  not  used  in  any 
calculations.  Defining  an  assumption  cell  though,  is  a  Crystal  Ball 
requirement 

-  All  portfolios  are  on  the  ‘Control  Calcs’  page  -  if  additional  portfolios  are 
generated,  just  add  information  to  the  bottom  of  the  current  list 

-  Set  up  the  function  call  in  a  cell 

=do _p._xep{assum  cell, hazlist, percent _effective, control  Jr ackjiumber) 
where  assumcell  represents  the  cell  where  the  assumption  is  defined 
hazlist  is  a  range  of  cells  containing  the  names  of  the  hazards  affected 
by  the  controls  in  the  portfolio 

percent  effective  is  the  range  of  cells  containing  the  percent  effective 
on  its  corresponding  hazard. 

NOTE:  the  order  of  the  hazards  and  their  corresponding 
percent _effectiveness  must  be  the  same.  For  example,  if  hazard  10  is 
the  third  hazard  in  the  range,  the  percent  effectiveness  number  must  be 
the  third  one  in  the  range  of  percent  effective 
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Control  track  number  is  used  when  recording  the  raw  or  severity 
totals,  and  must  be  a  unique  integer  value  for  each  control  being 
tracked 

-  Define  the  cell  containing  the  function  call  as  a  forecast  cell 
3)  Post-simulation  information  gathering 

-  After  the  simulation  has  completed,  use  Crystal  ball  to  gather  desired 
information  (forecast  values  or  statistics) 

-  If  raw  and  severity  totals  were  tracked,  run  the  macro  ‘macro_get_totals’ 
to  retrieve  the  data. 
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Appendix  D:  Mathemati  cal  Program 


The  mathematical  program  used  to  generate  portfolios  was  implemented  in 
Microsoft  Excel  using  the  premium  Solver  add-in.  The  objective  function  simply 
maximizes  the  coverage  on  the  hazards  by  selecting  the  best  mix  of  controls  for  the  given 
cost  level.  Other  constraints  considered  are  listed  below.  The  problem  is  formulated  as 
follows: 

49  controls  67  hazards 

Max  ^  ^  eij  *  Xi  where  ey  is  the  percent  effectiveness  for 

*= 1  j 

control;  on  hazard^  and  x;  =  1  if  control  i  is 
selected. 

49  controls 

Subject  to:  ^a*  xt  <  Max  Investment  Costs  c;  is  the  cost  of  control  i 

i=\ 

xio  =  0  Control  10  is  a  no  longer  under  consideration 

X34  -  X44  <  0 
X25  +  X46  <  1 
Xi6  +  X20  <  1 
X20  +  X2i  <  1 
X24  +  Xi5  <  1 
X17+  X44  <  1 


C-34  Requires  C44  (C44  DOES  NOT  Require)C-34 
C-25  and  C-46  accomplish  same  function 
C-16  and  C-20  Accomplish  Same  Function 
C-20  and  C-21  Accomplish  Same  Function 
C-24  and  C-15  Accomplish  Same  Function 
C-17  and  C-44  Accomplish  Same  Function 


108 


Appendix  E:  Portfolio  R  e suits 


The  charts  in  this  appendix  show  the  results  of  the  project  in  tabular  format.  The 
portfolios  are  ranked  from  best  to  worst  expected  performance  score.  The  checked  boxes 
indicate  the  corresponding  control  is  included  in  the  portfolio. 


%  Deer  Cost 


PC15  67.9%  230.55 


PC14 


PC8 

PC154 


PC147 


PC  152 


■nn 

Him 


211.55 


173.55 

211.55 


154.55 


173.55 


154.6 

154.6 

116.6 


PC143 


26 

PC155 

27 

PC89 

54 

PC96 

55 

PC161 

56 

PC181 

Q012E3iae!QIE3HOE3OW3QE»l 
□BE'  3BBBBBHQE3HH£9ElEaHi 

mmiMEmummuaummaaai 
OQE3IQIE»aaniSIQBDI 
OE3QIOQQK3HOE3HHI3QQHI 
QQE3ESaBBBE»DE3HHSaDQV'  ’ 

□e  •  lamioHHiaaQH 
mi  lOQinsonoBBii 
mamwmmmmmamummmawt 

OQQ13I1IOI1E  JOBli 
IQOIIIQIQIQBU  a  -  B1 
BBBBBBBBBBQiaHHiaOBHI 

mmauuuuumuaaMumauui 

sbbbbb  bbbb  120  as 

BBBBBBQflg»£3BM«ia£3Ei« 

mummuummmummuumaamt 
E3E»lll2l£«iaUIOBBIl 
iQinniSBIBnOBQU 
■■■■■QIIIIIBilllBllimi 

^■HiEiaKlilildiKiaBOaBBBQBOHOQHHeiaOOl 

pBBBBBBBI3BiBli3BBHHBiBil 

saBBBBflBttBHQQtfa  JQ.'-’BI 
PQEBBinQflBBBBQBBBI 
giilBBBOBUOHHHiaQDBII 
MBHBBBBBli 

|BBOBE3Hg3QHHOiaS3Ht 
IBBBBEH£' ■ •  *1 

gOBBBBHHaHHHHHM 
iBBaBMaaEMiaiaBiai 
SBQIflIBaBQBQI 
IpiBBBMiamMM 

Hobbbhohbmoboeii 

Ibbobqhohhbiobom 

gBBBBQBMMHHHElEaH 
||E3Qfl^HHQHHHHHBil 

mzmmmw&mMmwMmt 

■■KiBflQHilBHHQiaEaBI 
IBBBBBBBBQBHBaBMHHHn 

|BBBBBflBBaa£3M£lia£3n 

.  IBBBQBBB  S  IBBII 

^uuuuQuuumuuumuMEa 

EiiiiBBBBBHIlE3HHE3I3iSHI 

Bbbbbbbqbohheihhhhhhi 

QE3BBBBOBBHBE3HHBI3BBI 
£3E3BBBBOflBHi3MMilEiS3BH 

auummmummuumuuuuumi 

BEHBBQIIBBE3BBBBE3BI 

BBOBBBOBi»Hi3BHBflnBMI 

sQmummuumMumuumuumi 

E3ElBBBBOBQIf<  SBIBIBBOn 

OQBBBBBBBHBBHHBBBiai 
QEBHnQBIQBBIBflEll 
BBBBBBBBBHHBMISSiH%;f§9 
OQBBBBBBBHBiiafl 


PC7 


PC5 


PC92 


53  56. 


PC6 


PC97 


PCI  56 


PC93 


116.1 

154.6 

~ 

154.55 

135.6 

133.5 

117.55 

■Kzai 

1BHI 

BHBBI 

iBktUI 

— 

97.6 

115 

133.5 

B 

204 

115 

_ 

78.6 

fl 

79.6 

133.5 

PC68 


PCI  82 


PC141 


PC2 


PC59 


mm 


area 


C98  5 


PC35 


114.5  45.7 


79.6  65.6 

7^6  66T 

133.5 


133.5 


65.0 

I  79.6  I  64.1 


44.3 


114.5 


■■BHBEaBHBiaHBHBiaEl 

MBHQHiaBMiaElEaHBiaO 

■■■■■■SIQElBBiBHOHia 

■BBBBBQBBEBE3BOBD 

■■DHBBOOniEHB 

BBBBBBQBElBflBBBflQ 

■■■■■■laQQBHUO 

BIB!  IBOBOIBBIIB 
■■■BIBIHaOBOBmaHH 

umuummmMammuuamm 

119BBBBBBB1E3BOI  I 

BMBHmiMUiaE»iau| 

■BBiiaa— aiajj 

■■■BE3BQBBBBBBBBH 
BBBBBBBHBBBBBBig 

■BBBBBBBBBBBBBli 
BBBBBBBBBBBBBBH 

bbbbbbbbbbbbbbII 
■■bbbbbbmibbbbbI 

■■■■OE3ElE3S3£lMBaBB| 

■■■■■■BBBBB  IBI 
■BBBBBBBBBBBBBI 
■■■■■■■iiiinoi 
uuauuumuumuaMmt 

■BIBBBBBBBBBBBM 

■■BIBIHOHI3HHHElS5| 

■■■■■■■aaOHKSBBl 

muummmaumummmm 

— ■■■■■■■■BUMP 
■BBBBMBBBBBBB 
■■■■■BM£3BOBBB13 
■■■■■■■BBBIBIB 

■■■■''«  I 

BBlBFiBlBBBBlBiBi 
BflflBBBfl^fltjOBfl^H 
BBBBBBBBBBBBBBI 
BBBBBBBBBBBBBBI 

BBBBBBBBBilBBBBH 
BBBflBBBBBflflBBflll 
■IIKlIiimOIIH 
■■BHHBHaHBHBai 

■■■■HBiaBElEiMBHI 

UUUUMaUEmMUUUUi 

gaim«Hag 

BB—li 


PC164 

i  46.6% 

95 

49.1 

PCI  03 
PC88 

46.4% 

46.4% 

152 

95.1 

30.6 

48.8 

■01 

■01 

pim 

. uumm\ 

iohhioi 

MB®Wm 


noooi 

mumm 


ioi 

mm 

non 

mol 

mm 

IE3H 

mm 

IGOI 

igoi 

IIM 

11301 

■□■I 

■301 

■■131 

1 1301 

IOGI 


■■on 

IQOOE3I 

iooooi 


ioooi 


113 

114 

PC1 10 

IK3BBB 

45 

.4 

115 

PC25 

ikubbeeb 

K 

.6 

116 

PC39 

jgwwBKIif 

.1 

lOOEHl 

IBBBBI 

IOOOOI 


modooi 

■■00001 

IBBBBBI 

IIODDII 

■■DOOOI 


■BaBoBBSol 

■MIHO—Bi 

iobqbbbbobi 
bbbbqbqbbb 

IBBSBQBBBBBBBI 

■bbbbobqbbbqi 

yimnuamm 

mumuwum 
ummmuumm 
pilannaQi 

IpaiBiillj 
moiaoDiilll 

IOBQBBBBBBGBBI 

HHIQIQIIQII 

IOBOBQBBBBBBBI 

MMmiai— i 
nmaBiomm 

■DBBDDBBBBI 

iiniDiiianiai 

IOBQBQBBBBIIBBI 

IDBOBQBBBBOBOI 

IBBBQBBBBBBDBI 

ibbbqbobdbbboi 

IQBQBOBBBBBBBI 

IBBBBBBBBOOBBI 

IBBBBBilBOOBBBI 

ibbobbbbdbbbdi 

IBBOBBBBOBBBDI 

IBBBBBOiliiBBBOl 

IQBOBOBBBBOBBI 

IDBOBOBBBBDBBI 

IBBBBBBBQBOBB1 

IBBBBBOOBBBBOI 

■■■■■■OBDBBBOI 

IBBBBBOOOBBBBI 

IBBBOBBBDBBBBI 

IQBQBQBBBBBBBI 

IBBBOBODBBBBDI 

IBBBBBEIBQBBBGI 

IBBBBBBOOOBBBI 

IBB11BOBBBBBBBI 

IBBBOBBBOBBBBI 

IQBQBOBBBBBBBI 

IBBBBBQBBBBBQI 

IBBBBBI3BDBBBBI 

IBBBBE3BOOBBBBI 

IBBQBBOBBBBBai 

1BBBBGBBBBQBBI 

IQBQBOBBBBBBBI 

IBBBBBOOOBBBBI 

■BOBQBBBB 

IBBBBBBBOBBBB 

lOB^BOBBBBQBH 

■■■■■■OBOBIM 

■■■■OBBOOBBBBI 

BBBBBBBOOOBBBI 


110 


j _ _ _ : _ ;  Control  j 

Rank  J~~~  %  Deer  Cost  feenefit/Cos|  1  1  2  I  3  1 4  I  5  1  6 1 7  I  8  [  9  1 1 1 1 1 2 1 1 3 1 14 1 1 6  j  1 8 1 1 9 1 22 1 23 1 25 1 27 1 28 1 33 1 34 1 35 1 37 1 38 1 40 1 41  [  42 1 43 1 44 1 46 1 47 1 48 1 49~ 

119  PC30  41.7%  114.05  36.5  X  X  X  XXX  W: 


5  _ 51 

6  33 

5  34 


Ol 


IOE3BI 

IE3BE3I 


101 


IBQBBBBBBOl 

IBBBBBflBOOl 

■obi  mum 

■BBBBBBBBI 

■OBBBBOflOl 


IBBOMI 

IBBBHI 

IIIOIOI 


IBBBI 


IBBI 


im 


33.6%  57.1 


mat 


EHB0BBBBBBB3I 

pi 

_ ■ . . . Bbi 

oaoQimgan 

EZmOBflflflflOBBI 


PC176 


PC78 


77  41.2 

95.6  33.1 


95.55  32.5 


166  18.2 


IQI 


131 


PC125  27.3% 


1131 


PC52  24.9%  38  65.5 


PC71  24,5%  _ 3 

PC73  24.2%  3 


pi 
Eai 

ii 

■01 

nFPBBQBBBBBBQI 

_ _ _ _ _ _ _ _3| 

□I 


1BMBBM 

mmuumumi 

IQQnnnHHI 

IBBBBBBBI 

MnnflQQQI 

innnnQQQI 

IBBBBBBBI 

IBBBBBBBI 

MQnnSinfll 

IHE3HHUHI 

non 

IBOB 
I1BI 

man 

ibbIbIIII 

IBBBBBBBI 


IQI 


IQI 


IBBBBBBBQBBBBn 
■1BBBBBBBB1BI 
■BBBBBBBBBBBI 
IBBBBBBBBBBBBBI 
IBBBBBBBBBBBBBI 
■BBOBSBBBBfll 
IBBBBBBBQBBBBn 
IBBBBElBBBBflfllBI 
BBBBB3BOI  IB 
BBBBBBB1BBB 
IBBBBBBBBBBBBBI 
IBBBBBBBBBBBBfll 
IBBBBBQBBBHBBBI 
BBBBB3B3BOBBBI 

innnnnonQHiniini 

IBBBBBBBBBBBBBI 

IBBBBBBBBBBBBBI 
IBBBBBBBBBBBBfll 
IBBBBBBBBBBBBBI 
IBBBBQQBBBBBBBI 
■BBBBBBBBB  I,  i»S 
■BBBBBBBBBBBI 
IBBBBBBBBBBBBBI 
IBBBBBBBBBBBBBI 
■BBBBBBBB3BBI 
WBBBBBBBBBBiae 
IBBBBBBBBBBBBBI 
IBBBBBBBBBBBBBI 
IBBBBBOI 
IBBBBBBBBBBBBBI 
IBBQBBBBBBBBBBI 
IBBBBBBHBQBBU 
IBBBBBBBQBBBBn 
■■BBBlBBBnnBBI 
innQnnnnannnnni 
BBnoninBnHM 
innnnnnnnn 
mumaumumu 
nnnnnonon 
iiinEjnnnnnn 


IQI 


ioi 


IQI 


IQI 


38 

38.1 

t 

IQI 


inai 

mm 

IQQI 

IQQI 


IQI 


IQI 


IQI 


IQI 


IQI 


IQI 


IQI 


Jj 

■ 

nnnnn 

■ 

1H 

■ 

iaoQonn 

Ql 

■ 

nnnnn 

IQI 


IQI 


mi 


in 


Appendix  F:  Mann-Whi  tney  U  Test 


The  Mann- Whitney  U  test  is  used  to  determine  the  statistical  significance  between 
two  portfolio  expected  severity  distributions. 


Ho:  The  population  relative  frequency  distributions  for  A  and  B  are 
identical  (means  are  equivalent) 


Ha:  The  two  populations’  relative  frequency  distributions  are  shifted  in 
respect  to  their  relative  locations  (means  are  not  equivalent) 


Ua 


m(m  + 1)  TT_ 
=  nmi  +  — - - --Wa 


Test  Statistic: 


z  = 


Ua-(—) 

2 

{mmfa  +  riz  +  l) 

12 


Where  nq  =  number  of  observations  in 
Sample  A 

n2  =  number  of  observations  in 
Sample  B 

WA  =  ranked  sum  for  Sample  A 

Rejection  Region: 

Reject  Ho  if 

Z>Za/2°rZ<-Za/2 

(two-tailed) 


Assumptions:  Samples  were  randomly  and 
independently  selected.  Ties  in  observations  are 
handled  by  averaging  the  weights. 

The  results  of  these  tests  are  shown  on  the  following  page.  The  charts  are  read  in  a 
row-column  manner.  For  example,  to  check  the  result  of  comparing  Portfolio  163  with 
Portfolio  177,  find  the  last  occurrence  of  the  two  in  the  left  hand  column  and  follow  that 
row  over  until  it  intersects  the  column  of  the  other.  If  the  code  SD  is  in  the  box,  the  null 
has  been  rejected  and  the  two  means  are  statistically  different.  Otherwise,  Not  SD  in  the 
box  indicates  ul  =  u2.  If  the  weight  of  unit  readiness  equals  0.25,  comparing  Portfolio 
177  and  Portfolio  163  reveals  they  do  not  have  the  same  mean. 
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Results  from  varying  the  weight  of  Unit  Readiness: 


Weight  of  Unit  Readiness  =  0.25 


Statistics  Rank  PC162  PC163  I  PC177  I  PC179  I  PC180  I  PC181 
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Appendix  G:  Severity  Si  mutation  Visual  Basic  Code 

This  Appendix  contains  a  listing  of  the  Visual  Basic  for  Applications  code  created 
for  this  project. 


'Severity  of  Losses  Simulation  Visual  Basic  Code 
‘Global  Declarations 

’max  eu's  pre-determined  used  in  unit  readiness  severity  calcs 

Public  Const  max_train_exec_eu  =  0.2 

Public  Const  max_unit_fat__eu  =  0.125 

Public  Const  max_unit_mag_inj_eu  =  0.45 

Public  Const  max_unit_unavail_acft_eu  =  0.03 


'num  constants  used  for  arrays 
Public  Const  num_reps  =  410 
Public  Const  numcases  =  110 
Public  Const  num_hazards  =  67 
Public  Const  numcontrols  =  6 
Public  Const  draws_in_a_rep  =10 
Public  Const  num  units  =  26 


'B  number  of  reps 

’total  number  of  UH-60  cases 

'  UH-60  hazards 

’  number  of  controls  being  recorded  as  sim  runs 
’n  draws  in  a  rep 

'estimate  of  number  of  UH-60  battalions 


Public  cas_matrix(l  To  num  cases,  1  To  numhazards)  As  Double  'stores  haz  casualty  matrix 

Public  cost_matrix(l  To  num_cases,  1  To  numjiazards)  As  Double  '  cost  haz 

Public  freq_matrix(l  To  num_cases,  1  To  num_hazards)  As  Double  '  freq 

Public  env_matrix(l  To  num_cases,  1  To  num_hazards)  As  Double  '  same  as  freq  matrix 

Public  accident_data_matrix(l  To  num_cases,  1  To  1 1)  As  Double  '1 1  data  points 

Public  weights(l  To  14)  As  Double  ’weights  for  14  sev  funcs 

Public  std_err(l  To  num_reps,  1  To  num  controls)  'keeps  track  of  std  err 

Public  sev_totals(l  To  num_controls,  1  To  4)  As  Double  'keeps  track  of  sev  scores 

Public  raw_totals(l  To  num  controls,  1  To  3)  As  Double  'raw  -  fatalities,  $,  AC 

Public  run  number  As  Integer  'sim  run  number 


Function  do__a_rep(cbvar,  hazlist,  percenteff,  control  num) 

'cbvar  is  just  a  dummy  variable  to  force  crystal  ball  to  recored  this  as  a  forecast 
'hazlist  is  the  range  of  cells  containing  the  hazards 

'  percent  eff  is  that  portfolio  (or  control)  effectiveness  on  the  corresponding  hazard 

Dim  x(draws_in_a_rep  - 1)  As  Double  'stores  the  n  draws 
Dim  haznum(67)  As  Integer 


'fill  the  matrices  if  necessary 
If  weights(l)  =  0  Then 
fill_matr  ice  sandarray  s 
End  If 

'if  crystal  ball  isn't  running  a  sim,  this  cell  is  0  and  the  function  bombs  out 
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'the  conditional  takes  care  of  that  situation 

runjiumber  =  Worksheets("Control  Calcs").Cells(6,  1 1) 

If  run_number  =  0  Then 

GoTo  quitit 

runjiumber  =  1 

End  If 

number_of_hazards  =  hazlist.Count 
’  search  through  the  hazard  list  to  identify  which  hazards  are 
'  currently  being  evaluated 
For  j  =  1  To  number  j)f_hazards 
haznum(j)  =  fmdhaznum(hazlistQ) 

Next  j 


'do  n  draws 

For  i  =  1  To  draws  jn  a  rep 

num  draws  =  get_rand_number_of_draws(i)  'number  between  5  and  21 
x(i  -  1)  =  rand_draw(num  draws,  haznum,  percent_eff,  control_num,  numberofhazards) 
Next  i 

avg  =  Application.  Average(x) 

std_err(runjmmber,  control jium)  =  Application.  StDev(x) 
doarep  =  avg 
quitit: 

End  Function 

Function  ran  ddraw(n  umber jiraws,  haznum,  percent  eff,  control  num,  number_of_hazards) 

Const  numjinits  As  Integer  =  26 
Dim  unit_fat(num  units)  As  Double 
Dim  unit__train_exec(num_units)  As  Double 
Dim  unit  mag  inj  (num  units)  As  Double 
Dim  unit jmavail_acft(num jmits)  As  Double 

'automate  matrices  population 

actual  num  accidents  =  0 


For  i  =  1  To  number  draws 

'  number  of  draws  represents  the  number  of  accidents  in  100,000 
'flight  hours.  Each  draw  is  one  accident. 

'  initialize  reduction  totals 
casreduction  =  0# 
cost j*eduction  =  0# 
freq_reduction  =  0# 
envreduction  =  0# 

'  pick  the  accident 

casenum  =  Int((l  10  *  Rnd)  +1)  '110  cases 
'pick  the  unit 
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unit_num  =  Int((num_units  *  Rnd)  +  1)  ’  numunits  declared  at  top 

'  determine  %reduction  for  the  hazard  combination 
'  for  each  hazard  in  the  hazard  list,  see  if  it  contributed  to 
'  the  current  accident  and  record  amount. 

For  j  =  1  To  number_of_hazards 
If  haznum(j)  o  0  Then 

cas_reduction  =  cas_matrix(casenum,  haznum(j))  *  percent_eff(j)  +  cas_reduction 
cost_reduction  =  cost_matrix(casenum,  haznum(j))  *  percent_eff(j)  +  cost_reduction 
freqreduction  =  freq_matrix(casenum,  haznum(j))  *  percent_eff(j)  +  freq_reduction 
env_reduction  =  env_matrix(casenum,  haznum(j))  *  percent_eff(j)  +  env  reduction 
End  If 
Next  j 

*  does  the  accident  occur 

’  Controls  (or  hazard  combination)  may  have  reduced  the  probability  of  occurrence. 

'  if  this  amount  of  reduction  is  significant,  the  accident  is  skipped, 
happen  =  Rnd 

If  (happen  >  freq_reduction)  Then 

actual_num_accidents  =  actual_num_accidents  +  1 
'  if  it  does  occur,  adjust  case  data  accordingly  and  add  to  running  totals 

’get  fatalities  for  the  current  accident 

currentfatalities  =  (1  -  casreduction)  *  accident_data_matrix(casenum,  1) 

’add  to  total  fatalities  for  current  block  of  100,000  fliying  hours 
fatalities  =  currentfatalities  +  fatalities 

’perm_tot_dis  commented  out  because  there  are  none  in  this  project 

'permtotdis  =  (1  -  casreduction)  *  accident_data_matrix(casenum,  ???)  +  perm_tot_dis 
’  permanent  partial  disabilities 

'get  current  permanent  partial  disabilities 

current_ppd  =  (1  -  cas  reduction)  *  accident_data_matrix(casenum,  2) 

'add  to  totals 

perm_part_dis  =  perm_part_dis  +  currentppd 
'days  in  the  hospital 

current_dh  =  (1  -  cas_reduction)  *  accident_data_matrix(casenum,  3) 
days  hospital  =  cuirent_dh  +  days  hospital 

’  accident  data  matrix  has  accident  costs  (meaning  no  injury  or  damage)  in  the  4  spot 

current  accident  costs  =  (1  -  cost_reduction)  *  accident__data_matrix(casenum,  4) 

'total  accident  costs  are  in  the  5  spot  and  include  injury  and  damage  costs  as  well 
total_costs  =  (1  -  cost  reduction)  *  accident_data_matrix(casenum,  5)  +  total  costs 

’  unit  training  execution,  metric  used  is  Accident  occurrence  *  class 
'  a  Class  A  is  worth  3,  B  ->  2,  C->1 
'  this  is  per  unit 

’  first  the  pertinent  raw  accident  data  is  transformed,  then  the  new  data  is  used 
'  to  determine  the  'new'  class  of  the  accident 
imit_train_exec(unit_num)  =  unit_train_exec(unit_num)  + 
get_class_number(current_accident_costs,  current  fatalities,  current_ppd) 
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'fatalities  in  the  unit  -  the  unit  was  randomly  drawn  above 
unit_fat(unit_num)  =  current_fatalities  +  unit_fat(unit_num) 

'  magnitude  of  injury  -  categorical 
'  looks  like  a  mess,  but  it  really  isn't. 

'  use  the  'new'  data  points  to  determine  the  magnitude  of  injury 
newcat  =  get_mag_inj_cat(current_ppd,  current_dh) 

1  take  the  maximum  occurrence  of  magnitude  of  injury  for  each  unit 
If  newcat  >  unit_mag_inj(unit_num)  Then 
unit_mag_inj(unit_num)  =  newcat 
End  If 


1  an  aircraft  is  unavailable  to  the  unit,  either  because 
’  it  was  totalled,  or  is  gone  for  repair 

*  the  data  isn't  adjusted  with  frequency 

’  because  it  has  already  been  determined  that  the  accident 
1  has  occurred.  Cost  reduction  is  used  to  represent  the  amount  of 

*  reduction  in  severity  for  repair  hours. 

unit_unavail_acft(unit_num)  =  unit  unavail  acft(unit  num)  +  determine_loss((l  -  cost  reduction)  * 
accident_data_matrix(casenum,  6))  +  accident_data_matrix(casenum,  7) 

'  these  blocks  adjust  the  environmental  spillage  estimates 
'  after  the  adjustment,  the  get  cat  function  re-categorizes 

'  the  data  -  using  the  max  category  as  a  proxy  to  severity  of  env  damage. 

» 

’  Assumes  worse  case  of  both  water  and  soil  being  affected 


f  =  get_cat((l  -  env_reduction)  *  accident_data_matrix(casenum,  8)) 
h  =  get_cat((l  -  env  reduction)  *  accident jiata_matrix(casenum,  9)) 
o  =  get_cat((l  -  env_reduction)  *  accident_data_matrix(casenum,  10)) 

'only  bother  checking  the  spillage  if  there  was  some 

If  f  >  0  Then 

1  if  there  was  some  fuel  spilled,  record  it  if  is  the  max  occurrence. 

If  f  >  fuels  Then 
fuels  =  f 
End  If 

If  f  >  fuelw  Then 
fuelw  =  f 
End  If 
End  If 

If  h  >  0  Then 
If  h  >  hydraulics  Then 
hydraulics  =  h 
End  If 

If  h  >  hydraulic  w  Then 
hydraulicw  =  h 
End  If 
End  If 
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If  o  >  0  Then 
If  o  >  oils  Then 
oils  =  h 
End  If 

If  o  >  oilw  Then 
oilw  =  h 
End  If 
End  If 


End  If  '  end  of  the  if  block  of  -  did  it  happen? 
Next  i '  go  draw  the  next  accident 


For  j  =  1  To  num_units 
’get  expected  utility  for  unit  variables 
‘this  sums  the  severity  of  each  measure  for  each  unit 
’after  the  loop,  the  average  is  taken  -  implying  each  unit  is  equally  weighted 

'  two  if  statemets  for  unit  traiin  exec  because  of  S  curve 
If  (unit_train_exec(j)  <  3  And  unit_train_exec(j)  >  0)  Then 

train_exec_subtotal  =  train_exec_subtotal  +  (valueE(unit_train_execQ,  Worksheets(”Training 
Execution”  ).Range(”K3  5”),  Worksheets("Training  Execution”).Range(”K37”),  ”1”,  Worksheets("Training 
Execution”).Range(”M35”))  *  0.5) 

Else 

If  (un  it _tram__exec(j )  >=  3)  Then 

train  exec  subtotal  =  train_exec_subtotal  +  ( valueE(unit_train_exec(j ),  Worksheets("Training 
Execution").Range("K37"),  Worksheets(”Training  Execution").Range("K39”),  "Y\  Worksheets(”Training 
Execution”).Range(”M36"))  *  0.5  +  0.5) 

End  If 
End  If 

'make  the  function  call  to  get  unit  fatalities  severity  only  if  there  was  one  in  the  unit 
If  unit_fat(j)  >  0  Then 

unit  fat  subtotal  =  unit_fat_subtotal  +  valueE(unit_fat(j),  Worksheets(”Lives  Lost  in  Unit 
(Morale)").Range(''K35"),  Worksheets("Lives  Lost  in  Unit  (Morale)”).Range("K37M)?  ’T', 
Worksheets(”Lives  Lost  in  Unit  (Morale)").Range(”M35")) 

End  If 

If  unit_mag_inj(j)  >  0  Then 

unit_mag_inj_subtotal  =  unit_mag_inj_subtotal  +  valuePL(unit_mag Jnj  (j ) ,  Worksheets("Magnitude 
of  Injury").Range(nK35:K38”),  Worksheets(” Magnitude  of  Injury").Range(ML35:L38”)9 
Worksheets("Magnitude  of  Injury ").Range(”K3  8")) 

End  If 

If  unit_unavail_acft(j)  >  0  Then 

unitunavailacftsubtotal  =  unit_unavail_acft_subtotal  +  valuePL(unit_unavail_acft(j), 
Worksheets("Unavailable  Aircraftn).Range(”K35:K40"),  Worksheets("Unavailable 
Aircraft”).Range(”L35:L40”)5  Worksheets(”Unavailable  Aircraft”).Range(”K40")) 

End  If 

Next  j  '  end  of  unit  expected  utility  calcs 
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'  average  the  unit  scores 

train_exec_eu  =  train_exec_subtotal  /  numunits 
unit_fat_eu  =  unit  fat  subtotal  /  numunits 
unit_mag_inj_eu  =  unit_mag_inj_subtotal  /  numunits 
unit_unavail_acft_eu  =  unit_unavailacftsubtotal  /  num_units 

'  run  the  measures  through  the  severity  functions 
'  severity  multiplied  by  the  weights  in  the  function  calls 

'casualties 

c  =  casualty_severity(fatalities,  perm_tot_dis,  perm_part_dis,  days  hospital) 

'  unitreadiness 

ur  =  unit_severity(train_exec_eu,  unitfateu,  unit_mag_inj_eu,  unit_unavail_acft_eu) 
'costs 

tc  =  cost_severity(total_costs) 

’  environmental  damage 

env  =  env_severity(fuels,  hydraulics,  oils,  fuelw,  hydraulicw,  oilw) 


’take  the  comments  away  from  the  next  lines  to  store  raw  totals  for  a  run. 
raw_totals(control_num,  1)  =  fatalities  +  raw_totals(control_num,  1) 
raw_totals(control_num,  2)  =  total  costs  +  raw_totals(control_num,  2) 
raw_totals(control_num,  3)  =  actual  num  accidents  +  raw_totals(control_num,  3) 


sev_totals(control_num,  1)  =  c  +  sev_totals(control_num,  1) 
sev_totals(control_num,  2)  =  ur  +  sev_totals(control_num,  2) 
sev_totals(control_num,  3)  =  tc  +  sev_totals(control_num,  3) 
sev_totals(control_num,  4)  =  env  +  sev_totals(control_num,  4) 


’  rand  draw  =  casualties  +  costs  +  unit_readiness  +  env  damage 
*  rand_draw  =  unit  mag  inj  eu 
randdraw  =  c  +  ur  +  tc  +  env 
End  Function 

Function  casualty_severity(f,  ptd,  ppd,  dh) 

’  if  the  value  functions  change,  these  HARDCODED  cells  need  to  be  updated! 

’  the  IF  statements  are  there  to  save  run  time  when  the  parameters  are  0 
'  all  information  for  severity  functions  start  in  cell  K35 
If  (f  >  0)  Then 

fatalities  =  valuePL(f,  Worksheets("Lives  Lost”).Range("K35:K36n),  Worksheets("Lives 
Lost").Range("L3  5  :L36"),  Worksheets("Lives  Lost").Range("K36")) 

End  If 

'  no  permanent  total  disabilities  in  this  project 
'If  (ptd  >  0)  Then 
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’  perm_tot_dis  =  valuePL(ptd,  Worksheets(M Total  Disabilities").Range("K35:K37"),  Worksheets("Total 
Disabilities"). Range("L35:L37"),  Worksheets("Total  Disabilities").Range("K37")) 

’End  If 

If  (ppd  >  0)  Then 

perm_part_dis  =  valuePL(ppd,  Worksheets("Partial  Disabilities").Range("K35:K36"), 

W orksheets("Partial  Disabilities").Range("L35:L36"),  Worksheets("Partial  Disabilities").Range("K36")) 
End  If 

If  (dh  >  0)  Then 

days_hospital  -  valuePL(dh,  Worksheets("Tinie  Incapacitated").Range("K35:K36"), 
Worksheets("Time  Incapacitated").Range("L35:L36"),  Worksheets("Time  Incapacitated").Range("K36")) 
End  If 

casualty  severity  =  (fatalities  *  weights(l))  +  (perm_part_dis  *  weights(2))  +  (days  hospital  * 
weights(3)) 

End  Function 

Function  costseverity(c) 

’  no  if  statements  here  because  there  is  (almost)  always  a  cost 
'cost  severity  function  is  in  millions  of  dollars 

costs  =  valuePL(c  /  1000000,  Worksheets("Total  Costs").Range("K35:K36"),  Worksheets("Total 
Costs"). Range("L35:L36"),  Worksheets(" Total  Costs").Range("K36")) 

costseverity  =  costs  *  weights(4) 

End  Fimction 

Function  unit__severity(ute_eu,  uf_eu,  umi_eu,  uua  eu) 

’  normalize  all  of  the  expected  utility  based  on  the  predetermined 
’  maximums  -  would  need  updating  if  severity  functions  change 

a  =  (ute_eu  /  max_train_exec_eu)  *  weights(5) 
b  =  (uf_eu  /  max_unit_fat_eu)  *  weights(6) 
c  -  (umi_eu  /  max_unit_mag Jnj  eu)  *  weights(7) 
d  =  (uua  eu  /  max_unit_unavail_acft_eu)  *  weights(8) 

unit_severity  =  a  +  b  +  c  +  d 
End  Function 

Function  env_severity(fs,  hs,  os,  fw,  hw,  ow) 

*  severity  functions  are  different  for  water  and  soil 
’  only  make  funtion  call  when  >  0 
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If  (fs  >  0)  Then 

fuelseverity  =  valuePL(fs,  Worksheets("Fluid  Spills").Range("K35:K40"),  Worksheets("Fluid 
Spills").Range("L35:L40”),  Worksheets("Fluid  Spills").Range("K40")) 

End  If 

If  (hs  >  0)  Then 

hydraulic_severity  =  valuePL(hs,  Worksheets(  "Fluid  Spills").Range("K35:K40"),  Worksheets("Fluid 
Spills,,).Range(,,L35:L40"),  Worksheets("Fluid  Spills").Range("K40")) 

End  If 

If  (os  >  0)  Then 

oil_severity  =  valuePL(os,  Worksheets("Fluid  Spills").Range("K35:K40"),  Worksheets("Fluid 
Spills").Range("L35:L40"),  Worksheets("Fluid  Spills").Range("K40")) 

End  If 
’soil  total 

total  =  (fuel_se verity  *  weights(9))  +  (hydraulic  se verity  *  weights(lO))  +  (oil  se verity  *  weights(l  1)) 

If  (fw  >  0)  Then 

fuel  severity  =  valuePL(fw,  Worksheets(nFluid  Spillsn).Range(nK45:K50M)?  Worksheets(nFluid 
Spills’').Range("L45:L50"),  Worksheets(nFluid  Spillsn).Range("K50n)) 

End  If 

If  (hw  >  0)  Then 

hydraulic  severity  =  valuePL(hw,  Worksheets(MFluid  Spills").Range("K35:K40"),  Worksheets("Fluid 
Spills,,).Range(,,L35:L40n),  Worksheets(MFluid  Spillsn).Range(nK40”)) 

End  If 

If  (ow  >  0)  Then 

oil_severity  =  valuePL(ow?  Worksheets("Fluid  SpillsM).Range(nK35:K40'f)5  Worksheets("Fluid 
Spillsn).Range(nL35:L40M),  WoiksheetsC'Fluid  Spilis").Range("K40")) 

End  If 

'wate  total  +  soil  total 

env_severity  =  (fuel_severity  *  weights(12))  +  (hydraulic_severity  *  weights(13))  +  (oil_severity  * 
weights(14))  +  total 

End  Function 

Function  findhaznum(haz) 

'crude  way  of  doing  it,  but  it  works 
findhaznum  =  0 

For  j  =  3  To  69  '  number  of  hazs,  starting  in  col  3 

If  Worksheets("Haz  Env  Matrix").Cells(4,  j)  =  haz. Value  Then 
findhaznum  =  j  -  2 
Exit  For 
End  If 
Nextj 

End  Function 

Function  get_cat(a) 
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'get  environmental  damage  category 


If  a  <-  0  Then 
getcat  =  0 
GoTo  exitit 
End  If 

If  (a  >  0  And  a  <  1)  Then 
getcat  =  1 
GoTo  exitit 
End  If 

If  (a  >=  1  And  a  <  2)  Then 
getcat  =  2 
GoTo  exitit 
End  If 

If  (a  >=  2  And  a  <  10)  Then 
get_cat  =  3 
GoTo  exitit 
End  If 

If  (a  >=  10  And  a  <  20)  Then 
get_cat  =  4 
GoTo  exitit 
End  If 

If  a  >=  20  Then 
getcat  =  5 
End  If 

exitit: 

End  Function 

Function  get_mag_inj_cat(perm_part_dis,  days_hospital) 

'magnitude  of  injury  category 

'from  interview  with  LTC  Semmens,  USASC 

If  permjpart  dis  >  0  Then 
get_mag_inj_cat  =  3 
GoTo  exitit 
End  If 

If  dayshospital  >  0  Then 
If  days  hospital  <  7  Then 
get_magjnj_cat  =  1 
Else 

get_mag_inj_cat  =  2 
End  If 
Else 

get_mag_inj_cat  =  0 
End  If 
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exitit: 

End  Function 

Function  get_class_number(current_accident_costs,  current_fatalities,  current_ppd) 

'accident  class  number 
'Class  C  accident 

'  has  to  have  no  perm  part  dis,  no  fat,  and  costs  <  200K 

If  current_ppd  <  1  And  current  fatalities  <  1  And  current  accident  costs  <  200000  Then 
get_class_number  =  I 
GoTo  exitit 
End  If 

'Class  B  accident 

'  involves  1  or  more  perm  part  dis  or  has  no  fatalities  and  is  less  than  1  Million 
If  (current_ppd  >=  1)  Or  (current  fatalities  <  1  And  current_accident_costs  <  1000000)  Then 
get_class__number  =  2 
GoTo  exitit 
End  If 

'  if  not  a  C  or  B,  must  be  an  A 
get _class_number  =  3 

exitit: 

End  Function 

Function  determineloss(mhr) 

'  see  if  the  aircraft  is  lost  to  the  unit 

'  if  it  requires  more  than  40  hours  to  repair,  it  is  sent  away  and  therrefore  lost  to  the  unit 

If  mhr  >  40  Then 
determineloss  =  1 
Else 

determineloss  =  0 
End  If 

End  Function 

'This  routine  must  be  run  before  the  randdraw  function  will  work 

'Here,  the  data  is  dumped  into  the  matrices  that  are  referenced  in  the  rand_draw  function 

Sub  fill_matrices_and_arrays() 
runnumber  =  0 

'for  each  accident 
For  i  =  1  To  num  cases 
'for  each  hazard 
For  j  =  1  To  num  hazards 
'data  starts  in  row  5  and  column3 
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cas_matrix(i,  j)  =  Worksheets("Haz  Casualty  Matrix").Cells(4  +  i,  2  +  j) 
cost_matrix(i,  j)  =  Worksheets("Haz  Cost  Matrix'').Cells(4  +  i,  2  +  j) 
freq_matrix(i,  j)  =  Worksheets("Haz  Frequency  Matrix").Cells(4  +  i,  2  +  j) 
eny_matrix(i,  j)  =  Worksheets("Haz  Env  Matrixn).Cells(4  +  i,  2  +  j) 

Next  j 


'fill  the  accident  data  matrix 
'fatalities 

accident_data_matrix(i,  1)  =  Worksheets("  Accident  Data").Cells(4  +  i,  2) 
'permanent  partial  disabilities 

accident_data_matrix(i,  2)  =  Worksheets(" Accident  Data").Cells(4  +  i5  4) 
'days  in  hospital 

accident_data_matrix(i,  3)  =  Worksheets("Accident  Data").Cells(4  +  i,  5) 
'Accident  damage  costs 

accident_data_matrix(i,  4)  =  Worksheets(" Accident  Data").Cells(4  +  i,  6) 
'Total  Accident  costs 

accident_data__matrix(i,  5)  =  Worksheets("Accident  Data").Cells(4  +  i,  9) 

For  k  =  6  To  10 

'man  hours  to  repair 
'total  loss  of  acft 
'fuel  spilled 
'hydraulic  spilled 
'oil  spilled 

accident_datajnatrix(i,  k)  =  Worksheets(" Accident  Data").Cells(4  +  i,  k  +  8) 
Next  k 
Next  i 

'get  the  global  weights 
'fatalities  weights 

weights(l)  =  Worksheets(” Accident  Data").Cells(3,  2) 

'perm  part  dis  weight 

weights(2)  =  Worksheets("  Accident  Data").Cells(3, 4) 

'days  hospital  weight 

weights(3)  =  Worksheets(" Accident  Data").Cells(3,  5) 

For  k  =  4  To  14 
If  k  <  9  Then 

'Total  Costs  Accident  weight 
'Class  #  weight 

’Fatalities_Unit_Morale  weight 
'Magnitude  of  Injury  weight 
'Acft  Unavailable  weight 

weights(k)  =  Worksheets(”Accident  Data").Cells(3,  k  +  5) 

Else 

'fuel  soil  weight 
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’hydraulic  soil  weight 
'oil  soil  weight 
'fule  water  weight 
'hydraulic  water  weight 
'oil  water  weight 

weights(k)  =  Worksheets("Accident  Data").Cells(3,  k  +  7) 
End  If 
Next  k 

End  Sub 

Function  valueE(x,  low,  high,  monotonicity,  rho) 

'  Determines  the  value  score  for  an  exponential  value  function 
'  Kirkwood,  pg  81 
If  x  <  low  Then 
valueE  =  0 
GoTo  exitit 
End  If 

If  x  >  high  Then 
valueE  =  1 
GoTo  exitit 
Else 

Select  Case  UCase(monotonicity) 

Case  T 

delta  =  x  -  low 
Case  "D" 
delta  =  high  -  x 
End  Select 

If  UCase(rho)  =  "INF"  Then 
valueE  =  delta  /  (high  -  low) 

Else 

valueE  =  (1  -  Exp(-delta  /  rho))  /  (1  -  Exp(-(high  -  low)  /  rho)) 
End  If 
End  If 
exitit: 

End  Function 

Function  valuePL(x,  Xi,  Vi,  high) 

'  Determines  the  value  score  for  a  piecewise  linear  value  function 
’  Kirkwood,  pg  81 
If  x  >  high  Then 
valuePL  =  1 

GoTo  exitit 
End  If 

If  x  <  Xi(l)  Then 
valuePL  =  0 
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Else 
i  =  2 

Do  While  x  >  Xi(i) 
i  =  i+  1 
Loop 

valuePL  =  Vi(i  -  1)  +  (Vi(i)  -  Vi(i  -  1))  *  _ 
(x  -  Xi(i  - 1))  /  (Xi(i)  -  Xi(i  -  1)) 

End  If 
exitit: 

End  Function 

Function  fluid_cat_to_amount(category) 

'used  for  estimated  amount  of  haz  fluid  spilled 
Select  Case  (category) 

Case  0:  fluid_cat_to_amount  =  0 
Case  1:  fluid_cat_to_amount  =  0.5 
Case  2:  fluid_cat_to_amount  =  1.5 
Case  3:  fluid  cat  to  amount  =  6 
Case  4:  fluid_cat_to_amount  =  15 
Case  5:  fluid_cat_to_amount  =  30 

End  Select 

End  Function 


Sub  clear_totals() 

Dim  Message,  Title,  Default,  MyValue 

For  j  =  1  To  numcontrols 
For  k  =  1  To  4 
sev_totals(j,  k)  =  0 
Next  k 

For  k=  1  To  3 
raw_totals(j,  k)  =  0 
Next  k 

For  k  -  1  To  numjreps 
std_err(k,  j)  =  0 
Next  k 
Nextj 
End  Sub 

Sub  get_totals() 

'this  is  used  when  I  write  out  severity  breakdowns 
'  to  work 


'for  severities 
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Worksheets(" Severity  Totals").Cells(l,  2)  =  ’’Casualties" 

Worksheets("  Severity  Totals").Cells(l,  3)  =  "Unit  Readiness" 

Worksheets(" Severity  Totals").Cells(l,  4)  =  "Total  Costs" 

Worksheets(" Severity  Totals").Cells(l,  5)  =  "Environmental  Damage" 

’for  aggregated  totals 

Worksheets("Severity  Totals").Cells(l,  9)  =  "Fatalities" 

Worksheets("Severity  Totals").Cells(l,  10)  =  "Total  Accident  Costs" 

Worksheets("Severity  Totals").Cells(l,  11)  =  "Number  of  Accidents" 

For  j  =  1  To  numcontrols 
Fork=  1  To  4 

Worksheets("Severity  Totals").Cells(j  +  1,  k  +  1)  =  sev_totals(j,  k)  /  (num_reps  *  draws_in_a_rep) 
Next  k 

For  k  =  1  To  3 

Worksheets("Severity  Totals").Cells(j  +  1,  k  +  8)  =  raw_totals(j,  k)  /  (num  reps  * 
draws_in_a_rep) 

Next  k 
Next  j 

get_std_err 
End  Sub 

Function  get_rand_number_of_draws(dummy) 

’binomial  distribution  n  =  30,  p  =  .295 
num  =  0 
Randomize 
For  i  =  1  To  30 
j  =  Rnd 

If  j  <=  0.295  Then 
num  =  num  +  1 
End  If 
Next  i 

If  num  <  5  Then 
num  =  5 
End  If 

Ifnum>21  Then 
num  =  2 1 
End  If 

get_rand_number_of_draws  =  num 
End  Function 

Sub  get_std_err() 

’recorded  macro  used  to  print  out  stderrs 
With  Application 

.Calculation  =  xlManual 
.MaxChange  =  0.001 
End  With 

ActiveWorkbook.PrecisionAsDisplayed  =  False 
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Sheets("Std  Err  Wksheet").Select 
Cells.Select 
Range("C  1 3 ").  Activate 
Selection.ClearContents 
Range("Ar?).Select 
For  i  =  1  To  numreps 
For  j  =  1  To  numcontrols 

Worksheets("Std  Err  Wksheet").Cells(i  +  1  J)  =  std_err(i,  j) 
Next  j 

Next  i 

End  Sub 

Sub  macro_get_totals() 

i 

’  macroget _totals  Macro 

'  Macro  recorded  2/17/00  by  rgallan 


With  Application 

.Calculation  =  xlManual 
.MaxChange  =  0.001 
End  With 

ActiveWorkbook.PrecisionAsDisplayed  =  False 
Sheets(”  Severity  Totals").Select 
Cells.Select 

Selection.ClearContents 
Range("Al  ”).Select 
gettotals 
With  Application 

.Calculation  =  xlAutomatic 
.MaxChange  =  0.001 
End  With 

ActiveWorkbook.PrecisionAsDisplayed  =  False 
End  Sub 

Sub  macro _get_std_errs() 

t 

’  macro  get  totals  Macro 
'  Macro  recorded  2/17/00  by  rgallan 


With  Application 

.Calculation  =  xlManual 
.MaxChange  =  0.001 
End  With 

ActiveWorkbook.PrecisionAsDisplayed  =  False 
Sheets(nStd  Err  WksheetM).Select 
Cells.Select 
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Range("C  1 3").  Activate 
Selection.ClearContents 
Range("Al").Select 
get_std_err 

End  Sub 


Bibliography 


Aviation  Safety  Investment  Strategy  Team  (ASIST)  Brief.  1999. 

Banks  Jerry,  John  S  Carson  II,  Barry  L  Nelson.  Discrete-Event  System 
Simulation.  2nd  edition  .  New  Jersey:  Prentice  Hall,  1999. 

Bunn,  Derek.  Applied  Decision  Analysis.  New  York:  McGraw-Hill,  1984. 

Byrd,  Jack  Jr.,  and  Ted  L.  Moore.  Decision  Models  for  Management.  New  York: 
McGraw-Hill,  1982.  ' 

Clemen,  Robert  T.  Making  Hard  Decisions:  An  Introduction  to  Decision 
Analysis,  2nd  Edition.  Duxbury  Press,  1996. 

Crystal  Ball.  Computer  Software.  Decisioneering,  Inc.,  1996. 

Davison,  A.  C.,  and  Hinkley,  D.  V.  Bootstrap  Methods  and  Their  Application. 
New  York:  Cambridge  University  Press,  1997. 

Department  of  the  Army.  Accident  Reporting  and  Records.  Army  Regulation 
385-40.  Washington  DC:  HQDA,  1  November  1994 

Department  of  the  Army.  Army  Accident  Investigating  and  Reporting. 

Department  of  the  Army  Pamphlet  385-40.  Washington  DC:  HQDA,  1 
November  1994. 

Department  of  the  Army.  Risk  Management.  FM  100-14,  April  1998. 

Department  of  the  Army.  System  Safety  Management  Guide.  PAM  385-16, 
September  1996. 

Efron,  Bradley,  and  Robert  J.  Tibshirani.  An  Introduction  to  the  Bootstrap.  New 
York:  Chapman  &  Hall,  1993. 

Gleisberg,  Lieutenant  Colonel.  Personal  Interview,  2000. 

Hurry,  Donald  F.  Measuring  the  Impact  of  Programmed  Depot  Maintenance 

Funding  Shortfalls  on  Weapon  System  Availability.  Thesis,  Masters  of 
Science,  Air  Force  Institute  of  Technology,  AFIT/GOR/ENS/96M-03. 
1996. 

Jackson,  Jack  A.  Jr.,  Jones,  Brian,  L.,  Lehnkuhl,  Lee,  J.,  An  Operational  Analysis 
for  2025.  Maxwell  AFB,  AL:  Air  University,  1996. 


130 


Keeney,  Ralph  L.  Value  Focused  Thinking:  A  path  to  Creative  Decision  Making. 
Cambridge  Mass;  Harvard  University  Press,  1992. 

Keeney,  Ralph  L.,  and  Timothy  L.  McDaniels.  “Value-Focused  Thinking  about 
Strategic  Decisions  at  BC  Hydro”.  Interfaces.  22:  94-109  (1992). 

Kirkwood,  C.W.  Strategic  Decision  Making:  Multiobjective  Decision  Analysis 
With  Spreadsheets.  California:  Wadsworth  Publishing  Co,  1997. 

Microsoft  Excel  97  for  Windows.  Computer  Software.  Microsoft  Corporation,  1996. 

Parnell,  Greg  S.,  Jack  A.  Jackson,  and  Jack  M.  Kloeber  Jr.  “New  Techniques  for  Value 
Model  Development:  Lessons  Learned  from  Major  Value-Focused  Thinking 
Studies”.  Proceedings  of  the  International  Conference  on  Methods  and 
Applications  of  Multi-Criteria  Decision  Making.  286-289  (1997). 

Risk  Management  Information  System  Database,  http://www.rmis.army.mil 

Semmens,  Lieutenant  Colonel.  Personal  Interview,  2000 

Sobol,  I.  M.  The  Monte  Carlo  Method.  Moscow:  Mir  Publishers,  1984. 

Sperling,  Brian,  K.  A  Value  Focused  Approach  to  Determining  the  Top  Ten 

Hazards  in  Army  Aviation.  Thesis,  Masters  of  Science,  Air  Force  Institute 
of  Technology,  AFIT/FOR/ENS/99M- 1 6.  1 999. 

US  Army  Safety  Center  Mission  Statement,  http://safety.army.mil/ 

US  Army  Safety  Center.  Brigade  and  Battalion  Commander  and  Staff  Risk 
Management  Booklet.  6  January  1 999. 

Winston,  Wayne  L.  Operations  Research:  Applications  and  Algorithms.  California: 
Wadsworth  Publishing  Co,  1994. 

Warren,  Colonel.  Personal  Interview,  2000. 


131 


